API standardization: holy grail or not?
Designing technical solutions that integrate multiple systems to optimize the user experience—whether for current or prospect customers, or internal users, such as retail staff or customer care agents—is inherently complex. The complexity of each individual system, combined with the challenge of integrating them in a reliable and high-performing manner, increases exponentially as the number of systems involved in the solution grows.
When considering system integration, APIs naturally come to mind. Traditional technologies, such as web services based on SOAP interfaces, along with REST APIs, have long been essential tools for solution designers and integration engineers. Moreover, modern technologies, including decoupled and event-driven architectures, are being employed to develop more reliable and resilient solutions. Despite these advancements, designing an appropriate data model, to facilitate the necessary information exchange, always remains a fundamental and critical aspect of any integration design process.
Due to the lack of reference to previous implementations and the need for expedited deliverables, the design of data models and APIs often becomes a fast-track process. The result is:
- a diverse set of APIs implemented in various ways, yet sharing similar or identical contexts
- unreliable and scattered knowledge base, which cannot be used as future reference
- a repeated API design process starting from scratch, resulting in significant effort and cost.
This is where the standardization of APIs plays a crucial role in streamlining and simplifying the design of new integrations. There is no need to design APIs from scratch for every new integration requirement or to create new entities and fields in the data model without any reference point. Predefined API specifications provide a reliable framework that can be leveraged to efficiently build potentially any necessary solution.
To further streamline the process and reduce the burden on individual companies to develop their own API specifications, Open APIs provide a repository of standardized API specifications available to the broader community. This approach fosters a mutually beneficial relationship, where members contribute to the development of new specifications while also gaining access to existing ones, ready for use in their own designs.
But is this enough? Are all members of the community receiving all expected benefits of the Open APIs?
API standardization is not enough
Assuming the role of an API designer, each new feature, enhancement, or optimization of existing functionality generates a demand for solution design, which may include API design requirements. The API designer has access to an extensive repository of Open APIs and can utilize any of the available specifications to develop the required API designs. However, the question remains whether the existence of such a repository is sufficient enough to ensure the optimal quality of deliverables.
Interpretation
API specifications can be compared to societal laws in that they allow for varying interpretations among those involved in API designs. The semantics of entities and their fields often provide room for differing interpretations. For instance, even simple cases, such as picking the proper API field to carry a subscriber’s phone number, can lead unexpectedly to divergent results, even between only two API designers. As the scale increases and multiple API designers work with the same specification without alignment, discrepancies in their designs are likely to be inevitable.
Extendibility
The primary principle of Open APIs is their extendibility, allowing for the addition of capabilities and functionalities not originally envisioned by their creators. These APIs are designed with polymorphism in mind, which is essential for supporting flexibility and expandability. However, this flexibility can complicate the adherence to standardization rules and maintaining a low level of design diversity.
A government process to handle API extensions is already in place in Vodafone, through the API Guild community. Its main pillars are to
- track all extensions introduced by any new solution design requirement,
- review and assess that any new extension adheres to the principles of Open APIs and aligns with the context of the API to which it pertains,
- approve and publish the extensions to ensure they are accessible for reuse by all.
However, the degree of strictness and the scale of the community involved in the governance process are additional factors influencing the possibility of diverse parallel extensions, developed for similar purposes.
Deliverability
Any additional process within the lifecycle of a solution impacts its overall delivery velocity. The greater the distance of this process from the delivery teams, the higher the risk of unexpected overhead and potential delays. This principle applies to processes managing for instance the Open API extensions as well; such processes introduce additional complexity and overhead into the API design workflow.
Pressure for rapid deliverability frequently leads to a compromise in fully adhering to established principles and guidelines. This often results in the use of workarounds, where inappropriate entities and fields are selected in API designs to expedite delivery, rather than following the formal process of requesting an extension or seeking consultation from the community. While technical debts may be documented to track these workarounds, there is no guarantee that they will be resolved, keeping the degree of diversity high.
Journey APIs
There are often cases where standardized Open APIs fail to adequately support the increasingly complex logic of business requirements in a performant manner. While simplicity in business operations may be the goal, particularly to minimize technical complexity, achieving this can be challenging, especially in highly competitive business environments. Requests for extensions by API designers, to accommodate all possible business needs, often fall outside the scope of the involved standardized APIs and, in most cases, are likely to be rejected.
In these cases, the journey APIs become relevant, if not the only solution. The journey APIs commonly cover use cases where an orchestration of multiple standardized Open APIs must be implemented to
- provide a customized user experience for end users accessing typically through frontend channels, such as mobile applications or websites.
- avoid implementing complex logic on the frontend channels, ensuring it is encapsulated and managed behind the API instead.
- be performant, without multiple calls towards the backend systems
These APIs do not adhere to any specifications, unlike standardized Open APIs, with obvious impact in the standardization.
Compromises
General agreements on compromises for simplification may also be established. Entities or fields that are considered required, according to the Open APIs specifications, might
- not be required from business perspective, thus, no reason to invest effort in supporting them
- not be technically supported or require high effort or cost
As a result, API designs and implementations may vary between departments within a company or across local markets in a multinational company, depending on the compromises each of them has made.
Lack of automation
Absence of automation in various stages of the API design and implementation lifecycle may lead to additional discrepancies, often resulting from human errors.
- The manual composition of an API design, without the aid of automated tools, may lead to errors or typos in the usage of correct names and values for API resources and fields.
- Manual code generation, as opposed to using automated tools for code generation, introduces additional risk of errors in transferring fields along with their names and values from the design to the actual code.
API Governance
Addressing the aforementioned challenges is complex. The following paragraphs outline a rule of thumb, designed to enhance governance in API design, based on three core principles: guidelines, knowledge base, and community.
API design guidelines
Aligned with the principles of standardization in Open APIs, the API design guidelines encompass a set of rules that integrate theoretical API specifications with practical application of them to address business needs. This ruleset represents a practical approach to API design and should be regarded as an additional knowledge base, documented and readily accessible to all stakeholders.
The context of the API design guidelines is built on two major pillars, aimed at making the API design process as comprehensible as possible for designers:
- Rules of how to design APIs. They provide answers to questions like the below:
- Where can I find the specifications of the standardized APIs?
- Where can I find documentation of already designed APIs, so as to understand their context and seek for reusability in the new design?
- How can I understand which standardized API fits to the context of the new design?
- Where can I find help in further questions I might have?
- How can I design new journey APIs?
- Rules of how to document the API designs. They provide answers to questions like the below:
- Where do I document the new design?
- Do I follow a per API documentation or a per new feature/business requirement?
- Is there a document template I can use? What info I must include in the documentation?
The guidelines are not static. They are updated and adjusted to accommodate any optimizations, adapt to evolving needs, and reflect the gained experience of the designers.
Knowledge base
Core outcome of the API design activity is the documentation it generates or is required to generate, which serves as a reliable knowledge base for reference. As such, this knowledge base:
- serves as the reference for all stakeholders interested in existing designs, supports operational issues, facilitates onboarding of new team members, and more,
- acts as the reference point for any future implementations, promoting reusability in both design and implementation, thereby minimizing associated effort and costs.
Together with the API guidelines, the knowledge base creates an comfortable environment in which API designers can confidently understand how to proceed, know where to search, and what to look for.
Community and design review
Relying solely on individual responsibility to adhere to guidelines is not effective. Instead, a community approach is introduced to foster a collective sense of accountability and responsibility in adhering to principles and guidelines.
Accountability can be assigned to specific individuals who serve as design reviewers and act as guardians of the guidelines, or it can be applied to everyone involved in the API design process through a peer review approach. The maturity and scale of the community are key factors in determining the appropriate model. In new or larger teams, having designated design reviewers can lead to better outcomes. In contrast, experienced or smaller teams may find a peer review approach to be more effective.
Responsibility, on the other hand, pertains to everyone involved in the API design workflow. Each individual is required to adhere to the established guidelines and, through the design review process—whether involving designated design reviewers or a peer review approach—ensure that these guidelines are consistently followed.
The community mindset is critical for the success:
- each individual feels integrated within the team and has the confidence to seek guidance or assistance from team members.
- each individual feels empowered to contribute to the team's growth by helping optimize processes, adjust the API guidelines, introduce automation and new ideas, and more.
Please check also how Vodafone Greece has adopted these principles: API Governance : Part of Solution Architecture in Vodafone