Data Product Strategy for the enterprise

Photo by Lukas Blazek @goumbik from Unsplash

The concept of data product, although it is not new, is gaining more and more influence in the corporate world. For some time now, this approach has been used in technology companies where the product was the core business itself or a very important component, but lately this strategy is being transferred to traditional companies, where the aim is to promote innovation and the creation of new products or business models, through this type of strategy.

In this article I will try to review how a data products strategy could be implemented within a company, in parallel to a traditional strategy.

But first, let’s start with the basics:

What is a Data Product?

In 2012, DJ Patil, the one-time Chief Data Scientist of the United States, defined a data product as:

…a product that facilitates an end goal through the use of data.

This definition is quite general and extensive, as any application could be considered as such, when using data in this way. Over time, this definition has been further specified and is now as follows:

A Data Product is a service or application that helps business to improve its decisions, and whose primary objective is the use and process of data to carry it out.

As examples of data products in a company we could find:

• Public data aggregator: From different external sources (opendata, public services, etc.) external data is aggregated to internal data and exposed to be consumed by other applications or applications.
• B2B data sharing: Service to share data with other entities or companies.
• Price recommender based on machine learning techniques.
• Service for automatic damage detection using computer vision.

All these products impact, to a greater or lesser extent, on business improvement.

Types of products

We can classify products into different types:

• Raw data: Data service with minimal processing (quality, security, etc.), which is offered internally or externally to be used by other applications or products, but where the user has to further process this data in order to obtain value.
• Derived data: Data service with more advanced processing in which the data has been enriched, aggregated, new entities generated from the original, etc. and where the user can access and search for the improved information.
• Algorithms: In this case, what is offered is not data an algorithm to process it, which has been developed with data. The data is sent to this product or service, processed, and returned to the user. An example of this type could be an intelligent search engine.
• Decision support: there is more extensive information processing in the form of data representation or simple analytical models. In this case, the information is processed in such a way that helps the user make decisions.
• Decision automation: Advanced analytics techniques such as ML or AI are used and actions are prescribed to the user. In this case it is the product that makes the decision.

This typology, although not official, is the most widespread. Looking at the product types, we can see that it is possible to have several products that compose other products (for example, an algorithm that needs to be trained with data from another product that aggregates them from an opendata source).

Benefits

The benefits of a Data Products orientation, both technological and business, can be grouped into 3 main types:

• IT Scaling: Products move from being led and executed by IT to being led and executed by business units, which multiplies capabilities.
• Innovation Democratization: As these capabilities multiply, more resources can be dedicated to innovation, which helps to democratize it.
• Focus on business: With this orientation, the focus is on the final product and the resolution of business problems, and allows greater agility when it comes to iterating the product and testing it against the market.

Product-oriented organization

This type of organization provides greater speed and efficiency in the developments and a greater contribution of value to the business throughout the development of the product. With this type of approach, we move from a project oriented organization where:

• Teams are assigned to projects, and when finished are transferred to other projects.
• Deliverables are focused on the plan and what was previously agreed upon.

To a product oriented organization where:

• Teams are assigned to the product for the life of the product
• Deliverables focused on value, not on planning
• Teams have all the necessary capabilities, no dependencies from other areas

The benefits of this approach are:

• Increased efficiency and productivity by having teams focused on delivering value on a continuous basis.
• Faster decision making as business is involved in development
• Teams have long-term assignments which allows them to better develop their business knowledge and contribute more to product development.
• Flexible architecture that can be developed according to technical needs

Governance and methodology

Another important point to consider is how these products will be developed and their governance. For the development of these, there are already Agile approaches successfully implemented in the market that are specifically designed for product development.

There also needs to be a person responsible for defining, driving and implementing this type of strategy, coordinating all stakeholders.

To this, governance should be added, since it is important that there is a global vision of the products being developed. This type of governance has the following main advantages:

• It avoids the generation of similar products
• It helps the reuse of products
• Coordinates the evolution of products to avoid impact on other products or business areas
• Prioritizes the development of use cases
• Standardizes methodologies, technology, etc.

For these reasons, it is convenient that there is an area responsible for the development of this type of strategy and for the governance of the projects developed. This entity should be led by a figure such as Chief Product Officer, a C-level management committee with the capacity to drive this type of transformation.

Levers

In order to implement a product strategy, there are certain strategic and technological levers that can accelerate the transformation. To summarize, we could consider the following:

Product platform

When we talk about product platform, we are not talking about a specific platform, but about a technological data and application architecture that allows to manage the data, data and AI lifecycle and its applications, in the most automated way possible (we will go deeper into this topic in the following points).

This platform will provide the necessary automatisms, so that a team of business analysts and developers can create data products with the reliability of business applications, and be self-sufficient in obtaining resources, deploying components, monitoring performance and consumption, etc. This gives more responsibility to the product team in the development and evolution of the product.

DevOps

Of course, in the field of automation, a DevOps approach is the basis. As this is already well known and widespread in the market, we will move on to explain the next level of automation.

MLOps

As with DevOps, by this term we mean the practice of enabling collaboration and communication between everyone involved in the advanced analytics development lifecycle, from business users to data scientists to the operations people needed to deploy analytic models into production, which enables the ability to streamline the entire process and allows for frequent iteration in the development of analytic models.

DataOps

Focusing on the data domain, this practice is becoming more and more widespread. The approach is to streamline and automate as much as possible the management of the data lifecycle. To this end, we are working along the following lines:

• Implement a collaborative data strategy, implement an agile organizational model with a product orientation.
• Implementing agile methodologies in the development of data applications, applying minimum viable product approaches and continuous delivery of value to the business.
• Work on data governance (quality, security, etc.) and automated lifecycle management as much as possible, to facilitate the democratization of data.
• Also automate all aspects of data application lifecycle management and architecture.

FinOps

We also have the concept of FinOps. Very important especially in the cloud environment and in this product strategy, since we will give access to multiple teams in the use of cloud components, which can generate some cost decontrol. For this, FinOps seeks to give independence and responsibility to the product teams so that they can control consumption, optimize it, etc.

The advantages of this approach are:

• Everyone takes responsibility for usage and consumption, not just operations.
• Costs are identified by product, so they can be taken into account in each product’s business plan.
• The variable cost model of the cloud is leveraged and usage optimizations can be applied.

To do this, certain prerequisites are necessary:

• Teams need to collaborate.
• The generated usage and consumption reports need to be accessible and timely, in order to accelerate decision making.
• There must be a centralized team driving this type of strategy.

NoOps

This focus on automation ends up leading us to the concept of NoOps, where the IT environment becomes so automated that there is no need to have people dedicated to managing the software operation.

There are several considerations in this approach:

• It does not eliminate operations completely. There will always be a need to have an operations team, as all this automation involves its own evolution and maintenance. What is achieved is that the operations team does not have to intervene in the development lifecycle.
• It is viable in the cloud, but in hybrid and on-premise environments it becomes more complicated as these environments lack the necessary tools.

Cloud

The last lever is the use of cloud technologies for the implementation of a successful strategy around data products. In the cloud environment there are approaches and tools that are not found in on-premise architectures. The main ones are listed below:

• Commitment to efficiency: The cloud is the technology facilitates the adoption of an efficient strategy in the use of data, infrastructure and applications for the systematic transformation of your business with sustainable goals.
• Artificial intelligence services: Acceleration of the process of advanced analytics and industrialization of solutions applied to the operational and analytical world, thanks to state-of-the-art AI development tools.
• Security and compliance: Cloud services guarantee the highest degree of reliability on data governance for responsible data use, including privacy, control, transparency, and regulatory compliance.
• Automation: All components are API and IaC ready in order to facilitate their automated management, enabling agility in their use throughout the product lifecycle.

Conclusion

As we have seen in this article, there are levers that accelerate the implementation of a data products strategy. There are both technological and organizational challenges, where the latter are the most complex to implement, as they usually require a change within the company’s culture. But it is a necessary change, since it is accompanied by a change in the corporate strategy, which seeks to evolve the business by relying on technology in a more agile way.