Posts

Is Your Organization Prepared to Manage Dark Data?

The Business Value of Mining Dark Data in Azure Infrastructure

As organizations accelerate the pace of digital transformations, most are moving to the cloud. In 2019, 91% of organizations had at least one cloud service. But, 98% of organizations still maintain on-premises servers, often on legacy infrastructure and systems. At the same time, moving to the cloud is a given for organizations wanting to take advantage of new tools, dashboards, and data management. The global pandemic has created a prime opportunity for many to make that shift. That also means shifting data from old infrastructure to new. For most, it means analyzing, processing, and dealing with massive quantities of “Dark data”.

Most importantly, that dark data is considerable. In 2019, Satya Nadella discussed Microsoft’s shift towards a new, future-friendly Microsoft Azure. In it, he explained that 90% of all data had been created in the last 2 years.  Yet, more than 73% of total data had not yet been analyzed. This includes data collected from customers as well as that generated by knowledge workers with EUC (End-user computing, such as MSFT Office, email, and a host of other applications. As a result, the process of big data creation has only accelerated and (unfortunately) more dark data exists now than ever before.

As organizations make the shift to the cloud, move away from legacy infrastructure and towards microservices with Azure, now is the time to unpack dark data.

Satya Nadella discusses Microsoft’s shift towards a new, future-friendly Microsoft Azure

Dealing with (Dark) Data

The specter of dark data has haunted large organizations for more than a decade. The simple fact of having websites, self-service, online tooling, and digital logs means data accumulates. Whether that’s automatically collected from analytics and programs, stored by employees who then leave the company, or part of valuable business assets that are tucked away as they are replaced – dark data exists. Most companies have no real way of knowing what they have, whether it’s valuable, or even whether they’re legally allowed to delete it. Retaining dark data is primarily about compliance. Yet, storing data for compliance-only purposes means incurring expenses and risks without deriving any real value. And simply shifting dark data to cloud storage means incurring huge costs for the future organization – when dark data will have grown to even more unmanageable proportions.

Driving Value with Dark Data

Dark data is expensive, difficult to store, and difficult to migrate as you move from on-premises to cloud-hosted infrastructure. But it doesn’t have to be that way. If you know what data you have, you can set it into scope, delete data you no longer need, and properly manage what you do need. While you’ll never use dark data on a daily, weekly, or even monthly basis – it can drive considerable value, while preventing regulatory issues that might arise if you fail to unlock that data.

  • Large-scale asset replacement can result in requiring decades-old data stored on legacy systems.
  • GDPR and other regulations may require showing total data assets, which means unlocking dark data to pass compliance requirements
  • Performing proper trend analysis means utilizing the full extent of past data alongside present data and future predictions.

Dark Data is a Business Problem

As your organization shifts to the cloud, it can be tempting to leave the “problem” of dark data to IT staff. Here, the choice will often be to discard or shift it to storage without analysis. But dark data is not an IT problem (although IT should have a stake in determining storage and risk management). Instead, dark data represents real business opportunities, risks, and regulatory compliance. It influences trend and performance analysis, it influences business operations, and it can represent significant value.

For example, when Stedin, a Dutch utility company serving more than 2 million homes, was obligated to install 2 million smart meters within 36 months, they turned to dark data. Their existing system, which utilized current asset records in an ERP was only enabling 85% accuracy on “first time right” quotes for engineer visits. The result was millions in avoidable resource costs and significant customer dissatisfaction. With Synerscope’s help, Stedin was able to analyze historical data from 11 different sources – creating a complete picture of resources and creating a situational dashboard covering more than 70% of target clients. The result was an increase to a 99.8% first time right quote – saving millions and helping Stedin to complete the project within deadline.

Synerscope delivers the tools and expertise to assess, archive, and tag archived data – transforming dark data from across siloed back-ends and applications into manageable and useable assets in the Azure cloud. This, in turn, gives business managers the tools to decide which data is relevant and valuable, which can be discarded, and which must be retained for compliance purposes.

If you’d like to know more, feel free to contact us to start a discussion around your dark data.

 

Ixivault™ – Complete View & Control of Your Data in the Cloud

Ixivault™ – software to get on top of all your enterprise data

The cloud offers compute, memory and storage horsepower and efficient cost to get every bit of information extracted from data. But it needs new software instead of a lift&shift of traditional analytical software to really obtain the benefits. And then there is the question what data to transfer to the cloud? Data silos, structured and unstructured data, Dark Data all stand in the way of an easy transfer. GDPR exacerbates this as it introduces three main challenges that Compliance, Risk and Data Protection functions must deal with, and on which they base their advice to the LOBs and Executive leadership:

  • Does your cloud computing connect with the sensitivity of the data you entrusted or want to entrust to the cloud? If a cloud computing solution is chosen where data processing and/or storing are shared between enterprise customers, the risk of data leakage is present.
  • The question which law applies: again the choice of software or software platform determines if sovereignty principles about the physical location at which certain sensitive data is held can be met.
  • The externalization of privacy requires that no cracks exist between the contracts an enterprise makes with software or platform vendors, the services companies, and the cloud vendor.

These are real hurdles in taking maximum advantage of the speed, scalability, flexibility and cost efficiency of the cloud. Much of this potential is lost when an enterprise at best feels safe transferring only parts of their data. The holes in this ‘Emmental’ cheese of data gets even bigger when we realize that most enterprises have near 70% of Dark Data. (Satya Nadella at Microsoft Inspire 2019)

SynerScope Ixivault™

We propose to scan all content of the enterprise data and using the cloud to perform that in a safe way. For this purpose, SynerScope introduces its Ixivault™ on Microsoft Azure. The setup is entirely within the enterprise’s own Azure tenant through Azure marketplace. The loading to the cloud and bulk scanning happens there (called a vault). The unknown dark data is transformed into known data and silo-ed data is linked so that a grounded decision on its release for further use can be made. Data that cannot be released for wider use in the enterprise cloud tenant is deleted from the vault. Data that’s safe is published for BI, data science and domain expert’s use by different functional departments in the company. All the original data sets stay safely in the company’s data center.

SynerScope turns Dark Data into Bright Data, ready to be used by human combined with machine intelligence for extracting information and value. Our three main solutions Ixivault™, Ixiwa™ and Iximeer™ are designed to handle any type of data in any combination, fast and flexible. Unstructured text, image and IoT data can easily be linked with structured data from ERP, CRM and other operational systems. To facilitate widespread secure and safe working with the data we support the (self) publishing of data to analytic processes (human-machine) under full linkage to the Data Protection Impact Assessment (DPIA) requirements. Granular content-based access control, integration of masking and hashing functions ensure that no eyes will see, and no process will use any un-eligible data (NoXi principle).

To further service the compliance, risk and data protection functions of the company, our systems will log every touch point of the data. Humans and machines can be audited for having worked inside the boundaries of Standard Operating Procedures (SOPs) set with each Project-DPIA. Providing evidence for an always-appropriate use of data is efficiently supported in SynerScope.

Azure

We have built our solution specifically to operate in your own Azure cloud-tenant environment. As an enterprise you directly agree with Microsoft Azure on the SLA’s with regards to data security. Adding SynerScope doesn’t affect these SLA’s and publishing of data inside your enterprise is linked to your own Azure AD set up. SynerScope prefers an Azure set up where all data lives in the ADLS/Blob store (and in its original application source system). We believe data should remain open for use in different applications.

SynerScope is proud to be a partner of Microsoft. We continuously look to exploit the expanding functions of Azure modules. Our serverless architecture provides the flexibility to deploy in any enterprise’s Azure tenant; we gladly welcome you to discuss opportunities to take advantage of your own tenant architecture.

 

IxivaultTM Azure

SynerScope Solution

Flexibility and speed of the SynerScope solution is secured by our patented data scanning, matching and visualization technology.
All this we have developed in aid of your move towards a data driven architecture in the cloud, including full compliance and with a view to provide you with significant cost reduction.
The solution is tried and tested to full satisfaction by our clients in the financial & insurance industry & the critical infrastructure industry.

“SynerScope’s platform might best be described as a data lake management product that covers everything from automated ingestion, through discovery and cataloguing to data preparation.” (BloorInDetail – 2018)

SynerScope will bring you:

Cost Reduction

  • Data warehouse optimization / application rationalization
  • Optimized use of data, cloud data storage and your cloud data warehouse architecture
  • SynerScope as migration and data publishing tool
  • Broad use of data in the LOB by domain experts, citizen data scientists or data scientists proper.

Efficiency improvement IT-operations

  • Incident/ problem root cause analysis reducing time to repair
  • Reduction of incident and problems
  • Usage of historical data and archives

The SynerScope Solution:

  • Extracting and accelerating insight from data with patented products
  • Open technology on Microsoft Azure
  • Deployment via Microsoft marketplace
  • Stand alone deployment also possible
  • Is complementary to third party data cataloguing tools, and adds considerably to the ease of use of unstructured data
  • Security and compliance proof (traceable & transparent)
  • Fast deployment

SynerScope References:

  • Financial & Insurance industry
  • Critical Infrastructure
  • Government (safety regions and smart city)

Inspired by Inspire

Some take-aways from Microsoft Inspire 2019

What’s happens in Vegas should not stay in Vegas…. Microsoft Inspire is one of the major events for the global data community. It is exciting to learn about the developing vision of Microsoft as the leader in all things data for everyone.

From 14-18 July, 2019, Microsoft Inspire invited its partners from over the whole world to Las Vegas. As a Microsoft Partner, we were there to learn, network, do business and have fun with other partners and the Microsoft solution specialists, managers and senior leadership. We will share our experiences and thoughts with you in a series of blogs.

In his keynote Satya Nadella, CEO of Microsoft, covered the increasing intensity and impact of tech to society. Gaming, modern home and workplace, business applications, IoT devices, AI and Machine Learning…  this tech intensity changes the way we live and work. One thing is for sure: we have become data-driven and data will rule the world “That’s what’s leading to us building out Azure as the world’s computer. We now have 54 data center regions.” (Nadella Quote)

To facilitate the new world of data, Microsoft works on the creation of a ‘limitless data estate’ based on Azure Databases and Azure Analytics. Limitless in scale, variety and place: in the cloud and at the edge.

Nadella propagated the democratization of AI and of app development. AI will expand to the many sorts of apps, services and devices that we use in daily life. Consumers and workers will benefit from the help of their digital environment in their doing and decision-making.

All those emerging innovations need to be created. Nadella forecasts that 500 million apps will be built in the next 5 years, more than we saw emerging in the past 40 years. But who makes them? One of the growing pains the digital transformation our society is going through, is the increasing shortage of application developers.

Citizen development, by ordinary users who build and then share their applications, will help to meet this huge challenge. The tools for easy and no-code development are here to stay. This allows everybody to become a developer.

Big Data is Dead, long live Big Data AI.

Nadella’s keynote was also about the big numbers. Of connected devices (50 billion in 10 years time), of users (3,2 billion worldwide at the end of 2019) and – of course – of data. To illustrate the scale of the data explosion: 90% of all data is created in the last 2 years! And mind this: 73% of the data (old and new) still needs to be analyzed.

This leads to a growing ocean of dark data with hidden treasures beneath its surface. You only need to find them and bring them to the surface! And exactly that’s where Synerscope comes in: we can help to create value by accelerating and optimizing the safe and controlled exploration and exploitation of your data. As Gil Press noted in Forrester early July: Big Data is Dead, long live Big Data AI.

In our next blog we will explain how to deal with the ocean of dark data in a safe and cost-effective way. Stay tuned!

SynerScope Now Available in the Microsoft Azure Marketplace

 

Microsoft Azure customers worldwide now gain access to SynerScope’s Ixiwa and Iximeer products to take advantage of the scalability, reliability, and agility of Microsoft Azure to drive application development and shape business strategies.

Helvoirt, The Netherlands — February 7, 2019 — As an innovator permanently pushing boundaries in big data information technology, SynerScope today announced the availability of its products Ixiwa and Iximeer in the Microsoft Azure Marketplace, an online store providing applications and services for use on Microsoft Azure. SynerScope’s customers can now take advantage of the scalability, high availability, and security of Azure, with streamlined deployment and management.

“We are very happy to be part of the Microsoft Azure Marketplace. For us it signifies the next step in our development and confirms that we bring a much needed value to the market: where others focus on building the data lake, we focus on bringing that data lake to value by getting data, extremely fast, in front of people in as many facets of your organization as possible,” said Jan-Kees Buenen, CEO, SynerScope.

Sajan Parihar, Director, Microsoft Azure Platform at Microsoft Corp, said, “We’re excited to welcome SynerScope’s Ixiwa and Iximeer solutions to the Microsoft Azure Marketplace, which gives our partners great exposure to cloud customers around the globe. Azure Marketplace offers world-class quality experiences from global trusted partners with solutions tested to work seamlessly with Azure.”

The Azure Marketplace is an online market for buying and selling finished Software as a Service (SaaS) applications and premium datasets. The Azure Marketplace helps connect companies seeking innovative, cloud-based solutions with partners who have developed solutions that are ready to use.

SynerScope is an innovator: permanently pushing boundaries in big data information technology. We create new visions and next-generation solutions by working with multiple kinds and massive amounts of structured and unstructured data simultaneously, resulting in easy to use, affordable, and highly effective technology to support you when you need data the most. SynerScope helps you make important strategic and daily operational decisions and helps reveal new strategic business opportunities at a glance.

For more information, press only:

Sylvia Wijshijer, SynerScope, +31 (0) 6 2364 4933, sylvia.wijshijer@synerscope.com

SynerScope is happy to announce it’s new Microsoft Gold Status for Partners

As an innovator, SynerScope permanently pushes boundaries in Big Data Technology. We create new visions and next-generation solutions by handling massive volumes of multiple types structured and unstructured data.
SynerScope delivers easy to use, affordable and highly effective solutions based on the flagship products Ixiwa and Iximeer. In order to support you when you need data the most: when important strategic or daily operational decisions must be made, and to show you at a glance promising strategic business opportunities.

This Microsoft Gold Status Partnership for Cloud Platform, offers both parties the possibility to work closely together in order to offer great support, both technical and commercial, especially to its Azure clients and thus driving your digital transformation.

A Microsoft Gold-Certified partner is a company that has agreed to collaborate and establish a close working relationship with Microsoft. Having that title means the company who applied itself has earned the highest standards of Microsoft’s widely recognized partnership program.

Events

Microsoft Inspire 2019, July 14-18 in Las Vegas

Synerscope will be attending Microsoft Inspire 2019

Microsoft Inspire is Microsoft’s flagship partner event, where partners and Microsoft employees from around the globe meet to connect, learn, and collaborate to accelerate digital transformation and the success of our customers.

Microsoft Inspire is five days of learning, networking, fun, and a few surprises that kicks off each new Microsoft fiscal year. It features executive presentations, training, and networking opportunities—all focused on your readiness to win with the best platform in tech and connecting you with other partners to drive your business forward. Wherever you are in your journey as a partner, Microsoft Inspire offers ways for you to accelerate your success, making attendance a valuable investment in your future growth.