Data transformation: Stick and carrot
26th September 2016 | Catchy Agency
This week I joined Microsoft and Gold Partner, Automated Intelligence at the Art of the Possible event. Having spent a month inviting key players in both private and public sectors, I was keen offer further support and hear more about exactly what is possible in the world of data transformation.
The exponential growth of data has been well documented. We all know that the data storm is coming, right? But before you can plan for the future, you need to deal with all the stuff you’ve already got. Legacy data must be transitioned to a secure, future proof and digital solution to reduce cost, improve efficiency and ensure compliance. Again, we all know that.
The live demo at the Art of the Possible conference set out to process 50 million items during the course of the day. With regular dashboard updates, we witnessed A.I.Datastorm analyse the data to identify redundant, obsolete and trivial (ROT) data, de-dupe it and then categorise and transition it to a SharePoint environment. All in less than 6 hours. 5 years ago this would have taken more than 7 weeks. One stat that really popped for me was that as much of 70% of data stored does not need to be kept and another was that one public body (unnamed) estimated that around 26% of its IT budget was spent on storing unstructured data. That added up to £1.3billion. Get rid of the ROT, guys.
So we all understand that there is a bunch of stuff that has to be done to comply with regulations and come within budget. One North London council anticipates £30m saving over 5 years from the enablement of digital services. Brexit and new Data Protection laws coming into force in 2018 have massive ramifications for the banking industry. The opening of the Microsoft data centres in the UK is a game changer, allowing organisations more choice and control over compliance, accessibility and governance of their data. These are all related to things that organisations have to do.
However, it was also interesting to hear how the massive growth and diversity of services now available in the cloud also give financial organisations the ability to be innovative and deliver better value to customers to stay ahead of newly emerging competition. Undertaking data remediation in order to be compliant has a direct effect on organisations but there are also indirect effects, offering massive opportunities. Adopting new systems and new ways of working offers greater flexibility to employees. In turn, this can increase productivity, creativity and make organisations more accessible and attractive to new talent.
Alongside facts and figures, speakers gave insight on the practicalities of large scale transitions – just how easy is it to migrate 930 users and 830k files over one weekend? It was also made clear that each transition is unique and one size does not fit all. But anything is possible, and in working to achieve compliance and governance, you may reap additional and unexpected benefits.
Mark Godfrey – Chief Executive Officer at Automated Intelligence
Simon Cole – Chief Technology Officer at Automated Intelligence
Jeff Johnson – UK Azure Sales Director at Microsoft
Steve Sharpe – Head of Microsoft Alliance at LAN2LAN
Fiona Durham – Senior Consultant, Governance, Risk and Compliance at Automated Intelligence
Steve Durbin – Interim Head of IT at the London Borough of Enfield Council
Michael Wignall – National Technology Officer at Microsoft UK
Michael Sampson – Management Consultant and Author
Paul Donnelly – Head of Corporate Digital Services at NHS Education for Scotland
Submitted by Lucy Richardson