Coalesce has come and gone and with it there were a lot of fantastic surprises and developments that we want to share. Starting with us, Joon Solutions, being awarded the dbt Gold certification award. There were only 12 companies featured for a certification award and we are super proud to be one of them. Our engineers worked hard and earned the right to be named dbt experts.
What does this mean for you? Well, we know what’s going on in the dbt community and we are happy to share with you some great insights we’ve gleaned over the past week.
“Knowledge-creating” Company – Marco Sollie, COO
The world wide push for data is nowhere as human as in the dbt community. Many of our team, folks throughout our customer base and professional friends can actually find a place to discuss ideas and get feedback on their solutions in the dbt slack channel. In fact, Tristan Handy, the CEO of dbt labs, kicked off his key-note speech, reminiscing on the 2020 coalesce with just 120 folks planning to meet up live at that tim, but how taking that online expanded it to an audience of 8 thousand plus attendees. The way in which, the team at dbt has been able to keep data enthusiasts central at the heart of the experience is phenomenal and makes us proud as a partner.
Tristan goes on to refer to a 1991 hrb article, The Knowledge Creating Company,
“In an economy where the only certainty is is uncertainty, the one source of lasting competitive advantage is knowledge”.
This rings so true with how we operate here at Joon, whether it be in start-up engagements or at the enterprise level, it’s about enabling knowledge to created and spread easily in a governed and trusted way for our customers. It’s an exciting time for data, and an exciting time to be recognized as a crucial partner to the community at a heart of the change. We look forward to continuing to help you become the “knowledge-creating” company, whose sole business is continuous innovation.
“The dbt Semantic Layer” – Tuan Nguyen, CTO
Since its introduction last year at Coelesce 2021, many of us have been waiting for the release of dbt’s native support for the semantic layer. So what even is a semantic layer? A metric layer is where you define metrics (duh), a time series aggregation that supports one or more dimensions. For example, revenue is a metric that most companies would have. Businesses need to look at revenue across many dimensions, such as category, region, supplier, distributor, etc. Even though dbt is excellent at organizing and orchestrating data warehouses, it doesn’t support the definition of metrics until today.
The introduction of the semantic layer is definitely not something new. Many companies are doing it, even an outstanding job of it. Looker’s implementation of the semantic layer is the product’s selling point and why the tool has recently surged in popularity. One downside with Looker, though, is its price tag. Not every organization can afford to move all of its BI infrastructures there.
That’s where dbt’s semantic layer comes in. The company has done a phenomenal job of developing dbt and making it accessible to everyone. Just look at the size of this year’s Colesece. For companies that already had their data warehouses orchestrated by dbt, it makes so much sense to have the metrics layer there too. Your data team will then be able to centralize most steps in the transformation process in one place and truly build a single source of truth for data. Morevover, dbt’s semantic layer is BI-tool agnostic, so you can use any BI tool and not have vendor lock-in.We can’t wait for the feature to be battle-harden and implement it for you. Check out the demo of dbt’s semantic layer here.
“The ROI on AE frameworks” – Jonathan Barrett, CEO
As an organization we are always trying to provide the highest value to our customers. This isn’t always easy to show when it comes to frameworks like what dbt provides with analytic engineering. During coalesce, there was a great talk on exactly this, ROI on dbt. David Jayatillake really goes deep and wide into the return on analytics engineering which is not always felt all the way through the data value chain. The reason this is and is very well explained in David’s talk, he considers it’s due to three areas of benefits which are Collaboration, Quality, and Value.
He starts by explaining the old methods of reviewing stored procedures for new team members to learn what the team had done beforehand (I remember this pain all too well). To get a new person up to speed and be productive quickly was nearly impossible. Using an AE framework and its detailed data lineage and governance makes this a cake walk. It also adds a benefit of better business continuity. In the case of losing a team member it is easy for someone to pick up where the last team member left off using this framework. He also points out that the BI layer is simply a presentation layer and should primarily be used for simple transformations. Something we have preached at Joon for years.
Using dbt and these frameworks, data that is available, is reliable, and product ready. Stakeholders can rest assured that whatever data they operationalize while using these frameworks is as good as gold. He wraps up on Super Stakeholders, the ones who like to work with data themselves and have been around long enough to do so. They are now enabled to work with trusted data assets due to the work of their analytics engineers. All of the data they need is clean, governed and quality tested so they can get right into their analysis.
To quote one of our customers, Matthew Lister, Director, Data Strategy & Analysis – Psych Hub LLC,
Work flows that would have taken hours to complete because data needed to be massaged, checked, and manicured now takes minutes due to the frameworks that Joon helped put in place with dbt Labs.
Watch the Coalesce talk by David Jayatillake here, read his blog here.
“Refine the role of AE” Ha Le, Analytics Engineer Team Lead
Marielle Dado, in her talk on “I dont build models, I construct knowledge”, refine the definition of what an Analytics Engineer (AE) does. It is not about model with clean data, it constructs knowledge, so that it helps stakeholders with sense making and decision making, drive business impact.
“Boundary object” is a good concept to think of when we think of model: something that is recognized as everyone to be the same object, but how they use it might be different (to answer different questions of business). When we think of a model that way, we need to account for multiple questions and use cases, expand what is existing or update it with time. She uses example of building a payment model and how the concept of boundary object is reflected there.
If you are looking to deploy any of the above new features covered during Coalesce book some time to chat with us!
If you want to learn more or want to be an AE read up on how to Become an Analytics Engineer in 90 days with us.
Experience more on dbt and mordern data stack with us:
- Get dbt here
- Data Modeling Pitfalls and Where to find them
- BI Tools choice and its impact on model planning
- More articles on Modern Data Stack
Chat with us about your modern data stack project or Join us on the next dbt meet up.
