In this article, our CTO, Darren Timmins, explains the concept of Reverse ETL, which automatically transfers derived and enriched data back to operational systems to drive better outcomes for both enterprises and their customers. This article was first published in Digitalisation World in March.
Within the fast evolving IT industry, we need to ensure that the right data is in the right place at the right time to drive the best outcomes. Lots of data is collected and processed, but often key insight around individual clients never makes it back to the operational systems used at the point of engagement. An interesting way of making this happen in a standard and uncomplicated fashion, and one that CIOs ought to be investigating, has been coined Reverse ETL.
That term ‘ETL’ may take you back 20 years to the first wave of business analytics, which in those days was all about creating data ‘warehouses’. The clue was in the name: we would Extract promising-looking data from our operational and customer databases, Transform it (which usually meant a lot of cleaning up and rationalising of it into a more machine-readable form)and Loading it into one of our then new-fangled data warehouses or marts.
Data warehouses and lakes are now commonplace and have combined to provide vastly more advanced analytics platforms, addressing the data needs of the ever data thirsty organisation. Increasingly those platforms are cloud-based, offering a scale and resilience difficult to achieve or fund on-premise to all but the biggest corporates. Tools to move and transform data have also become simpler, with many SaaS solutions available; examples such as Fivetran and Rivery allow for a low-code approach to landing data in hugely powerful platforms offered by Amazon, Google, Microsoft and Snowflake.
The focus of these tools, as well as their on premise counterparts, is still to extract data from as many available data sources as possible (e.g. CRM, billing, web analytics systems and so on), conform it, and then potentially enrich it with data from third party providers such as public datasets (think ONS), before making it available to the data community. The difference with today’s high end tooling is that it reduces the amount of coding needed and can significantly reduce time to delivery; an age old Achilles heel of ETL. But with the dots joined and the wide range of tools available, a global view of your enterprise and the universe it exists in is open for analysis.
The challenge in many scenarios today is that vast analyses are performed and a huge amount is learnt about behaviours and practices. Much thought is given to these outputs, but remarkably few companies push the client-level insight back to the operational platforms where that data is often invaluable to front line staff in a range of scenarios, but especially when managing interactions with clients.
In essence, you need to complete the circle. And recently, a practical, low code way of achieving that has appeared: Reverse ETL. This has the ability to take data that’s been worked on by machine learning or analytical processes from the analytical platforms and automatically transfer the derived attributes and append them to client records, so that it sits with the people out there selling, processing faults, or doing other useful work for you.
It enriches the data so that, especially where there’s interaction with a customer, your agent is better informed, enabling better conversations and greater understanding that will hopefully lead to better outcomes. Finally, what’s also exciting about this is that until recently, being able to do anything like this was really just the provenance of Fortune 500-level companies. But if you’re using, say, Salesforce for CRM but have interesting derived data in Snowflake, Reverse ETL offers you a low code way of pushing that data back, where before it may have required an expensive engineer or consultant.
Obviously there’s a big business driver here: if you want an uptick in customer engagement or reduction in churn, or a more tailored customer experience and to achieve better retention, for example. A great place to start with this would be to check out providers such as Census, Hightouch, Grouparoo or Seekwell, or for those who already utilise an ETL vendor such as Fivetran and Rivery, these are now expanding their integrations to include Reverse ETL.
There is one big caveat, however, as most of the low code offerings are SaaS. Therefore, if you are still not on the journey to the cloud for analytics, or if you are a legacy organisation with a lot on-prem, while you can still do Reverse ETL it is going to be a lot more complicated and difficult to manage.
But if you are in the cloud and therefore more easily able to access these services, I’d recommend you do. We all operate in a data-driven world so creating actionable insights and getting your useful data out to the field, be that your sales force or a field engineer, is vital. Reverse ELT ensures that all parts of your organisation have the data where they need it, and when they need it, and that’s got to be worth doing—no matter what you call it.
.png)
Copyright © 2023 Intuita. All Rights Reserved.