Loosely Coupled - Federated API Management

Hello, hello.

Good morning, good afternoon, and good evening, everybody.

Welcome to another edition of Loosely Coupled, brought to you by Bridging the Gap.

My name is Karol Skrzymowski, and I'll be your host today.

And today, today we're hanging around APIs, but we're not diving into the problems of APIs themselves.

Today, we're diving into a bit of a different problem managing those APIs and making them lovely, secure, and then discoverable, and then understood by the organisations.

And what happens if we have too many of them?

What happens if we have too many API gateways?

What happens if we have distributed systems, and more, and more, and more, and SaaS solutions everywhere, and custom implementations?

What then?

So, that's going to be today's topic.

So, I hope everybody has a nice beverage at hand.

You have a nice jug of coffee and my trusty mug.

And joining me today is Adeeb Tahir from APIWiz.

Hey, Karol.

Field CTO.

Hello, Adeeb.

How are you doing this evening?

Doing well.

How about you?

As best as I can with a cup of coffee.

Nothing can go wrong now.

No, coffee is always great.

All right.

So, to start us off, because our audience knows me already quite a bit, tell us a bit about yourself.

Who you are?

What do you do?

What's going on?

Why are you even here?

Sure.

Yeah.

So, my name's Adeeb Tahir, and I'm the Field CTO at APIWiz, but if I had to tell a bit more about me, I would call myself a hardcore engineer.

So, my whole career, I've been in software engineering, starting out in the fintech space with Citipank and Citigroup, based out of Chennai in India.

And then I worked there for a couple of years, and post that, I got an opportunity to work with APIWiz.

Initially, it was all fintech systems back in city, but now with APIWiz, I get an opportunity to work on the API strategy and the API management aspect side of it across industry verticals.

So, that's one of the most interesting aspects of my current role here.

Right.

But basically, from big companies to, what, a startup?

You could say that.

Yes.

Yeah.

What drove you to go for a startup?

I mean, it's a change, definitely, right?

Yeah.

No, that's for sure.

Within an MNC, there's a different set of challenges in an operating model.

And what I wanted to also experience is the thrill of working in a startup, where you have a bit more control, you have more after changes.

Usually, within an MNC, there's a structured process where it takes a couple of months to implement pretty much any change.

But within a startup, I have a bit more freedom to explore, a bit more freedom to innovate, fail fast, and come back to base.

So, I think that thrilling aspect, as well as travel, picking on new challenging aspects of software, that's something you can only get in a startup, I would say.

All right.

Looking at it from a perspective, do you enjoy the startup more or the structured, slow pace of corporate world?

No.

No, I'm fully into a startup.

I prefer a very high-paced, challenging environment.

Okay.

Yeah, because the difference for everybody is a little bit different in that sense.

And I remember my switches between working in-house and working as a consultant, and that always was a bit of a trade-off there in terms of figuring out what do I want to do in my life and where do I want to be.

And working in a big company for a longer time, it feels like, oh, I do have actually some accountability over things that I deliver.

Whereas when I'm working as a consultant, it's like, yeah, I'm here.

Two weeks, I'm somewhere else.

And where's my accountability?

I don't see the fruition of my labour, right?

So, I suppose there are these kind of trade-offs between working in a big corporate environment versus a startup.

I never worked for a startup myself, but I assume that there are such.

All right.

So, jumping into the topic at hand, and we're talking today about, well, managing APIs.

Perhaps for those who are less accustomed with certain terms, let's start by defining what is an API management, because that's the base of our conversation, right?

Yeah.

So, it's a really heavy and interesting topic today.

So, just to start out with, API management essentially is the entire set of system of practises and operations and pretty much the strategy we use to build, operate and govern APIs, which are nothing but application programming interfaces that pretty much connect every application that we can think of today.

They power all the digital experiences.

So, it's very important that we design, build, test and govern and get them right.

Right.

So, if I'm looking at this, because you're looking at it from a productized perspective, but if I'm looking at it from a purely technical perspective of an integration architect, for me, an API management is three distinct things.

First of all, that's API gateway, which is the implementation of the proxies themselves.

Then on top of that, or next to it, it's an API portal where I have the definitions of those APIs and some little bit of a governance over access and testing and mocking those APIs to play around with them.

And then third is the actual API management part where we have security policies or any other policies surrounding those APIs around access to them, who can use them based on what credentials, et cetera, et cetera.

And that basically constitutes a piece of software that I would call an API management.

Yes.

It's about solving for those several components and processes across the design time, the build time, as well as the runtime.

Right.

Okay.

So, let's place that API management somewhere, right?

So, where do we find API managements?

Yeah.

So, if you think about API management today, at least let's take the lens of an enterprise, let's say one of the large enterprises, whether it's healthcare or telco or tech or automotive, pretty much whenever they take the concept of API management, they talk about how am I building my APIs today?

So, they'll always say that have an existing API gateway, I'm creating proxies, I'm managing the APIs around those.

But some of the other questions that come into API management is how do I design them or how do I build them today?

Or how do I enforce a governance aspect of it?

Who approves these APIs?

Who has a final say in saying this is the last approved change and no one can make a change or edit this API once it's approved and locked in?

So, you jumped straight into the management part there.

But let's step a little bit back.

If I'm thinking where do we find API managements?

Basically now, if you look at any SaaS offering, any API on the SaaS offering is basically fronted with an API management, pretty much.

We find API managements within companies' fronting systems.

There are very popular API management solutions fronting entire microservice ecosystems, like the swarm of microservices just being fronted by one piece of software, of course, instantiated just to unify the layer of APIs that those microservices expose.

And then we have also APIs somewhat on the perimeter of companies, giving them some sense of management and security over their connection to the outside world, the outside services, partners, whatever that may be.

So, we have different flavours in that sense.

Yes, that's true.

And then we have scale to it.

Yes.

In fact, just speaking about the variety of offerings out there, one interesting stat that I keep seeing from a time-to-time basis is if you take a look at the API landscape today, we have individual more than 2,500 plus these offerings scattered across the different stages of your API management.

So, you can see a lot of different individual point solutions as well as players trying to aggregate themselves into a platform.

And that gets really crazy over time because a number of platforms and number of point solution offerings out there, it's quite growing.

Hold on.

Give me the number one one more time because I'm not believing what I'm hearing.

No, you can quickly look it up.

It's well over about 2,000 plus vendors out there.

And in fact, there's a pretty neat, I think there's a website API landscape.io where you can see a clear breakup of, let's say, vendors at the design, vendors at the test, vendors at the, let's say, observability.

And you can see exactly how that vendors are broken up.

Okay.

What I see is like, yeah, okay.

I just Googled.

I'm shocked.

Yeah.

But they're probably, like you said, they're fragmented between different problems of the API space, right?

So, I remember somebody once telling me that Postman is an API management, right?

It's like, I'm sitting there and it's like, no, wait, Postman is a testing tool.

How is that an API management?

But then if you look at it from a perspective of managing definitions of APIs and mock services and tests of APIs, well, oh yeah, kind of is.

And they're kind of going that direction to become a full-blown API management at this point.

But it's quite shocking at times, which companies actually take a turn and go in a different product area.

And that sometimes is quite a surprise when doing market research.

But that number you said, that's quite a number of different solutions.

All right.

So, if we're looking at an enterprise, let's say whatever, telco, automotive, okay.

Usually those big corporate environments, they have somewhere between 20, 50 systems, however you define a system, completely independent applications that are governed by some sort of a business unit or used by some sort of a business unit to achieve some sort of a business goal, right?

Most of them really need to communicate with something else.

So basically they utilise some sort of integration quite possibly via APIs, right?

Some still use files with SFTP or managed file transfer, but let's leave them at that.

Okay.

Some of them are using APIs.

So if we're looking at an enterprise level, we have, let's say that 50 systems, let's say that's a large telco environment that's very feasible to have 50 different systems.

And nowadays, every one of those systems, custom or not, SaaS, custom, on-premise, in cloud has an API management, right?

So we're already somewhere here on a problem of scale, right?

So what problems of scale do you see at your perspective from this kind of setup?

Yeah.

So definitely, one of my go-to phrases is every sort of API management is a problem of visibility and governance, right?

So more number of APIs, I would call it as API sprawl,

because teams today, if you take a look at how teams operate within larger enterprises,

I'm sure that you've also experienced the same, that the teams work in, I would call it as silos,

or we pretty much see that they work on one particular functionality, this team is responsible

for a particular domain of APIs, and you might have 15 or 20 or even hundreds of other teams

working on different set of functionalities.

So, but do these teams actually, let's say, collaborate on certain integrations or applications and other functionality?

The answer is, it's very rare that the teams have that sort of central view of what the other team is building on, or what I already built to have a central catalogue of all my APIs that I've exposed, is it up to date?

One of the trending, or one of the most common things that I see is developers hate to document, that is universally accepted, that's the truth.

If you ask a developer, hey, can you write the documentation for this?

It's always at the back of the mind, we don't have an automated way of doing this.

Nine out of 10 chance I'm not going to do it, it's always an afterthought.

But these sort of challenges start when it works within a single team, yeah, everything's fine, right?

Everyone knows, hey, this is a set of four or five APIs, everyone knows what it does, how to access it, and some examples of how to use it.

But the moment you start exposing an API, right?

Let's say one team has to work with the auto management team, so that's when there's a central system of record, then you have to expose your data models, you have to expose your data to the other team.

And that's where it's challenging, because a lot of the systems within banks, let's say banks, telcos, they were built more than eight to 10, or 15 plus years ago.

So the people who built them are long gone, but who are the current team that's maintaining it?

Do they have a clue of what's the exact functionality of what they do?

But these are just some of the challenges that come in.

But I would say there's a massive, not just a skill gap, but also sort of like an organisational structure gap within a lot of the enterprises.

They don't know exactly how their API is, where they are, where they are running.

I think we see in the news today, there's a lot of shadow IT, or there's shadow APIs.

APIs that are running in a server that I don't have a clue about, but they're still consuming resources, adding to your cloud costs.

And that's exactly what escalates the problem at scale.

Right.

I remember situations working in large corporate environments, especially in corporate environments that are network of companies, rather than a single large company in a territory.

So there are lots of those that are spawned across the globe in a single territory.

It's not a single company.

It's just a few companies under one brand.

And then obviously those companies do not have proper cohesion over their IT.

So it happens to be that one part of the company is developing exactly the same functionality as the other part of the company.

They don't talk to each other because they're logically in two different, completely different branches of the company.

They're doing the same thing, exposing the same APIs, doing exactly the same logic because they need it.

They don't know about each other.

And then we have duplicates, and then we have all sorts of wonky stuff, even using the same data from the same source at times, but basically exposing two separate functionalities.

And it's amazing that these things happen, but these are not issues that are technical issues, basically.

This is Conway's law, pretty much.

So this is more of an organisational issue, which then drives a problem, which is driven by the lack of knowledge, lack of governance, lack of cohesion, lack of conversation at most times.

So basically like Conway's law says, the structure of the company, the structure of systems, and how many of them are there, is governed by how the communication happens in an organisation.

If it doesn't happen, it's going to be chaos, right?

So, okay, we're having this kind of a distributed mess of various systems, even duplicates in functionality, duplicates in some sort of duplicates in APIs.

We slap API management on top of that, right?

Well, on top of each part, let's say, because the scale problem is not that we have one API management, because then we wouldn't have a scale problem, but we slap multiple API management, because each of those teams is also responsible for their infrastructure under their app, and they're basically adding everything to it, right?

What then?

Yeah.

Most of the teams, they're not even located at the same geolocation.

They're distributed across the globe.

There are teams working in Manila, teams working in Dubai, teams working in India, and that adds to the challenge, right?

The teams have to collaborate on the same set of APIs, or most of the times, internal teams have their own wikis, their own sort of internal phrases or terms they use to refer to a certain process.

For example, fund transfer, they might call as InstaPay or FirePay, but in India, a team in the Middle East might refer to it as just fund transfer, just transactions.

So there's also a gap in context, right?

How you sort of refer to the same artefacts or the same processes in an organisation.

So I think that adds to the sort of ambiguity across the organisation.

So I would say that one of the biggest problems, or I would say the gaps that's missing today, is that central sort of like a dashboard control plane that tells you this is your entire API inventory, or this is your central API catalogue.

All the times where you're located can access it and find out this is exactly what my team builds.

This API is located here.

It takes a set of these parameters, it's connected to a certain system, and these are the set of access points, how to access and integrate with this API.

So I we're at the point of a knowledge problem.

We're lacking the visibility to have proper knowledge about what we actually have in our organisation.

So we're having our shadow ITs running per unit, whatever the unit may be.

We're having our shadow APIs running, duplicating efforts, duplicating functionality, because people don't know that that functionality already exists.

And then we have slapped onto them API managements, and we're creating a knowledge divide, because knowledge is still siloed, hence Conway's law.

Okay.

So apart from fixing Conway's law, which is called the reverse Conway's manoeuvre, so fixing how the organisation communicates, how can we properly support that visibility?

And I think now we're touching onto that governance aspect of API managements.

Correct.

That's right.

Yeah, that's when we sort of approach any type of governance, right?

Whether if you want to enforce a governance, or you want to enforce security, or you want to enforce any type of sort of checks and balances across your system, you first need to, irrespective of which vendor you sort of select, or whoever you consult, they always say, do you have a central inventory?

If not, let us first start with API discovery.

So discovery is something that can be done in so many different ways today, because that's a problem we've created ourselves.

Earlier back in the days of PIPCO, there was a fixed sort of way in which you could define a service flow, but now because of microservices, it's sort of become a mess, right?

So I could build a microservice in Java, Python, it doesn't really matter.

I still remember, that was somewhere in 2018, colleagues of mine were building, they called it the TIBCO radar, which was basically a repository crawler that went through the repositories and went through config files and identified and mapped all connection points.

So then we have a very large graph map through which we could just hop from point to point to different applications to see where the flow leads.

So we could map that, and that was an automated effort by then already, right?

But now given that we have all those systems that are way more distributed than, well, 2018, we already had a very distributed world, but people were still heavily in their own data centres, not that heavily in cloud yet.

Now we have everything everywhere.

Some companies have multi-cloud strategies to go anywhere, AWS, Google, Oracle Cloud and whatnot.

So what about that, right?

It's like, where's that governance now going with this?

And you say we should bring up the slide now.

All right, let's have it.

Slide time.

Tell me.

Yeah.

No, it's something through several conversations I've had, my team's had, and something that we generally just observe in the market, irrespective of what type of industry it is, right?

Whether it's healthcare, banking, or any industry, if you take a look at their actual sort of applications they're building, this is the most common pattern that we see.

It's always almost like an NTR model, right?

Where you have your final mobile app or your application access via a firewall, and you have an experience layer of APIs.

So you have that outer shell of all your comms or your Apigee or your AWS or your WS02, your gateway proxies, and then it connects back to your layer of system APIs, and of course, back to your core systems.

Now, one of the most challenging aspects here is if you take a look at any hop from your mobile app all the way to the core system, each layer is owned by a different team.

A separate operational team manages the gate proxies at Apigee or Kong, a separate team manages the backend system APIs, a separate team manages the integrations to the core system.

So that's something that we have seen commonly observed across several of the enterprises.

And that herein lies the challenge, right?

How do you actually get the teams working together?

The core system integration team understands a particular process in a different way.

The operational team that's managing the proxies on Apigee or Kong or the other gateways, they understand the process in a different way.

So they don't have that sort of cross-team visibility.

And that's where there's a big disconnect in terms of the architecture, dependencies, ownership, and how that whole integration exists today.

Again, back to the organisational issue of communication within an organisation.

By the way, I'm looking at this and it just hit me that you're actually using MuleSoft nomenclature for some names, but completely different than MuleSoft, does it?

Yeah.

Okay.

So let me try to digest.

Okay.

So yes, NTA definitely, if we add on top of that integration platform into the mix, then we have definitely more tiers than you have here on this diagram.

And that becomes, yes, this becomes very troublesome at times, depending, of course, what you want to address, but yeah.

And then we have multiple gateways, multiple various APIs addressing something.

Now you have them here very nicely divided by domains in the diagram, but most common scenario is that you don't really have that domain division on your APIs.

In a lot of organisations, they're just APIs.

They're not properly divided, addressed what they are, what they're supposed to represent.

And that's another level of problem itself.

Okay.

So we have our app, we have those NTAs, we have our APIs.

With API management, we're not going to change the APIs.

That's a completely different problem.

And we're not going to change the aspect of silos.

Okay.

So what now?

Yeah.

So given that sort of structure that we just now saw, Carol, so I think some of the most common observations that we see is because of that complex architecture and because of the lack of that central visibility documentation, we see that pretty much almost 30% plus of the APIs are misconfigured, right?

People don't know these APIs exist.

They might be using some outdated auth mechanism.

And we see that enterprises, because they don't see the visibility of that API, they end up spending 20% or more in terms of op-ecs.

Because even if an API is not being accessed, it's still, it's running, it's consuming resources on your, let's say your VM and your cloud, and it's adding to your overall cloud bill.

And heating up the server room, basically, somewhere in the world, right?

And on top of that, it's not being used, but it creates a security vulnerability.

Yes.

Yes.

The shadow ID is one of the most, I would say the most scariest, I would say, backdoor to your internal infrastructure.

That's always a nightmare to solve.

Yeah.

And one of the things that I found interesting is also, you know, there's a common disconnect between, I would say, the business and the IT divisions, right?

Because whenever we think about a new functionality that has to be delivered, the IT takes it up as a, I would call it as a project, let's say.

They start implementing the features for that.

You start pushing it in terms of, say, they take sprints, they release the functionality out to their environments, dev, test, 380, production.

But what happens is, let's say there's a bug found in production, right?

And this is a very common scenario.

So they roll it back, right?

They roll back to it.

But to the ID side of things, is there a cost associated to this?

No.

I just have to spend a bit more time, fix the bug, and again, push it to production.

But on the business side of things, they always take it as, hey, there is an inherent cost here, right?

I lost two weeks or three weeks of, you know, an effort.

Maybe I promised integration to a downstream consumer or a customer, or maybe now this leaves my integration effort in a bit of a peril because I need to sort of coordinate with other stakeholders.

So we see that there's a, because of this lack of the central or structured process of releasing your APIs or managing these APIs, well over 74% of your ID cycle is wasted, right?

There's always repetitions, creating duplicate, you know, efforts.

You might have already had this API, but because you didn't know about it, you created the API again.

So we see these type of patterns a lot, and it's a lot more common than we feel it is.

That's interesting.

Side note, please do send me sources for that later.

I love these kinds of numbers.

I always look for sources for these numbers.

I always love to share these sources because these are interesting studies.

I saw a very analogous study, and not analogous numbers for polling.

Standard approach for old systems where they cannot release events.

You pull data from those systems on an interval basis.

And somebody claimed over a presentation without giving a source that over 90%, 98% of polling attempts results in absolutely no data.

And that was a presentation given in 2024.

Turns out the study done for that was in 2013.

So it was quite old.

So it already was somewhat outdated.

Of course, you can quote, still quote it, but then what is the actual situation now?

That's a good question.

If I see these kinds of numbers, I always please ask for sources for those, because that might be actually a very interesting read to put that in context.

But let's say, I'll believe you on the word for the moment.

I'll read into it later.

We'll have a chat offline about sources, and I'll attach them to the stream, to the description later.

But basically, if we have a number of percent of repetition being, well, percent of waste through repetition being 74%, I mean, if somebody is watching this who's a CTO, or an IT director, or a manager in a IT unit, release management unit, that makes you think, how much do we actually waste in our context of our own organisations?

And this is a very interesting number, because then it seems that we could actually save quite a bit of money if we start governing that properly.

Yes.

That's right.

In fact, these cycles are not just attributed to pieces of code, but they could be, like you mentioned, you mentioned a good phrase, knowledge gaps.

So because of knowledge gaps, we take our release to the next environment, and it didn't satisfy the business or the functional requirement.

Because we didn't go through the process of stakeholder sort of iteration and feedback in the earlier stages, now we have to sort of bring it back and do it all over again.

So in order to miss that, sort of just make sure that we pass through those checks, it's always- If we try to map it to a cycle, the cycle that comes to mind in terms of governance is a ToGov cycle, right?

If you look at the ToGov cycle or any other governance of producing software cycle, a lot of the elements are not actually implementation, they're analysis, discovery, understanding the business, planning, design, operations, testing, deployments in various environments and so on.

So there's plenty to that cycle, more than just implementation.

That's for sure.

Correct.

Oh, there we go.

Yeah.

Yeah.

There's so many stages involved in the cycle, right?

If you have to, let's say, build or release a API that's reliable or that's compliant or functional or performant, you have to go through this entire sort of cycle with not just the tech, but also the business stakeholders.

And this is something that we see frequently within the enterprise.

And that's why I've sort of split this into two lenses here, right?

You have an API publisher who's building it out and exposing it as an API or a functionality to their clients or consumers, or it could also be internal groups.

And you have the consumer over there who's responsible for accessing the API, building the integration, or maybe even sharing it as your third-party integrator.

That's also something that we see a lot commonly used nowadays.

So pretty much the size of the API coin over here.

Yeah.

If we put this at this level of details, there's plenty to do around both sides.

And yeah, it's good that you put it this way because people tend not to think about this as a dependency.

Okay.

We publish an API.

Perfect.

Fantastic.

And we're done.

No, no, no.

There's also the consumer side, which is dependent on the API.

If that's a preexisting API and you're modifying it, there's so much to it, right?

Because we have consumers and the more consumers you have, the more dependencies you need to manage and it becomes a whole thing.

So then decommissioning of APIs, the whole life cycle is then humongous in terms of effort.

Yes.

And here's where an interesting terminology or concept comes into play.

We always ask consumer enterprise, how mature are your APIs or how mature is your API programme?

So ask us back saying, what is API maturity?

Does it mean that my consumers are accessing my API?

So my API is mature or my API is published in a portal.

That means it's a first class API sitting on a portal.

The consumer is accessing it.

But when we talk about maturity, it's a lot more than just is your API functional right?

Was it designed in the right way?

Is it extensible?

Can other teams access it, extend it, integrate with it?

Does it require a manual process to integrate with your API?

Because even in a lot of the conversations I have, existing in pre-sales and other integrating conversations, we always come across, even though you have a developer portal, it's a process of manually integrating the API, right?

The consumer sends an email, hey, I want to connect to this API.

Can you show me how to use it?

And then the provider has to manually insert some record in the database or generate the credential.

And that's a week or a two week process until the consumer can actually access the fourth version of the API.

So we see there's a lot of steps that need to go before we even say an API is mature, or it's ready for consumption, or it's a self-serve on its own.

So that's where we see that a lot of these aspects have to be taken care of in API management.

Yes, true.

But some of those APIs will require that process, because depending on the type of data you're dealing with and the, let's say, secrecy levels in certain organisations, like public entities, especially related to, let's say, police, border control, et cetera, the processes are there also for a reason.

Some of them are over-bureaucratised and then take the time with manual approvals, lots of red tape, which is unnecessary, but some of them are for good reason and could measure that they require approval to get to that API.

And this will probably differ from organisation to organisation, so that in that sense, if we look at it, then one might say that we need to redefine maturity per organisation.

Yeah.

But in the overall aspect of things, I would say, just to, I don't want to be too general here, but say when an API programme is mature or an API is mature, it means it's passed through the right guardrails, right?

While designing it, a centralised, sort of a standardised governance model, there were a couple of linting rules applied on that API.

I centralised my data models, accessed them from a, let's say, a data dictionary or a schema repository, and then I'm reusing them across the APIs.

I'm testing those APIs as well as integrating them in the right way, performing a scenario-driven test cases.

So if we were able to list down all these governance sort of practises or checks that have to be embedded, it's something that goes a long way into increasing or elevating the maturity of the API.

That's something that platform teams actively work on, right?

They want to make sure that instil that platform engineering within the organisation, or they want to make sure I have not just have a self-serve, but also implement that sort of like an ops.

Yes, we see API ops being used a lot these days.

So they want to establish that API ops framework within the organisation.

It usually all comes down to the actual adoption and utilisation of those, and we all want our software to be used in the end of the day.

And this will differ, again, from organisation to organisation, from piece of software to piece of software.

The question is then, can we actually have sort of a generic framework to actually measure the maturity of API?

Because that's a very compound problem, right?

If we look at this from the perspective of the previous stream, which was about data mesh, we were talking about data products.

Now, data products, that's something very specific.

If we define a data product, what is a piece of data, and then we serve it to a customer, then we serve it usually through APIs, right?

So is our API then mature because it's serving data products?

Well, then it would depend on the maturity of the data product itself in that sense, right?

So it's a matter of really putting guardrails also on the definition of that maturity first, and understanding what do we mean by mature?

Where is the target state of our maturity?

And can we define levels of maturity that we want to achieve?

And let's say now we're immature because we're having our APIs that resemble right now our relational database underneath.

So we have basically an API method per table, which is horrendous.

It's just a nightmare to use.

So let's say that's our maturity zero.

But then what drives us to maturity level one, two, three, four, five, let's say five is the maximum here.

And I'm curious to ask, when working on governance topics about APIs, do you have any sort of a framework or any sort of a classification method to classify maturity?

Yeah.

Maybe in the broader terms, Karol, I would say when it comes to assessing, let's say, the maturity of any enterprise's API programme, we start by examining first, do you have a central sort of API inventory?

How are APIs being accessed today, right?

Are APIs being sort of built bespoke or an ad hoc sort of development?

Or do your teams actually refer a common data model or a data dictionary or a repository and say that these are my existing models?

Can I sort of work these into a new functionality or extend my existing logic to expose new functionality?

So without having to, let's say, redo effort, or do I have a common or reusable artefact?

You know, a lot of times people say that in order to sort of promote or make your APIs more mature, you have to create that reusable artefact piece of it, right?

So that means rebuilding or redefining my data model.

And you mentioned earlier that, you know, you came across an example where the same API was duplicated.

So in a similar sense, you know, one of the examples that I've heard a lot of times is, let's say, within the telco space, you have a store locator API.

If you have seven versions of the store locator API that's created by, let's say, different teams within the organisation, how do you know which one to access?

Or which one is even actively being used?

Or if I have to go deprecated, which one do I deprecate?

Or which is the API in which I have to update the, you know, the parameter or the model?

I think that makes the first level of maturity such that you're just building functionality.

And then you start sort of going to higher levels of maturity saying that now I can standardise my API consistency.

I'm referring my data models from a common store or a repository.

So that means that I've sort of jumped from level one to level two.

And then level three means, do I have an automated design plan governance?

Whenever I push a change into my API, is it being automatically scanned against my organisational rule sets?

And that's going to differ from one enterprise to the other, right?

Even across industries.

For example, one good thing that I do see now is every industry or vertical has its own sort of like an organisational rule set, right?

You have, let's say for airlines, you have the IATA.

Then for banking, I think you have the sort of like the PSP regulations.

Similar like that, you have a lot of these standard operating models across the industries.

In Telco in Europe, at least in Europe, we have the TM forum standards, which are quite robust, extensive.

Yeah.

And also the sort of the Kamara Alliance, where I see a lot of these Telco operators coming together to build, you know, standards for networks, network APIs.

That's something that's really, you know, picking up pace.

That's something that really adds to the overall maturity.

And I would also say is, you know, people like to forget documentation, or they say, think of it as an afterthought.

But I would say a well, a well documented API also means that the API is a lot more ahead in terms of maturity.

That means your consumers are able to, you know, access it, understand it, they're able to integrate this API into their workflows and work streams.

So that's something that also adds to the maturity of the API, as well as your API is well secured.

Are they following the right set of updated or, you know, mechanisms?

Does it follow a systematic security, you know, protocols in place that your organisation enforces?

And this is something that I've seen differs across enterprise to enterprise, they might have their own set of, you know, security policies or practises.

But that's something that has to be enforced at each enterprise.

Right.

And then finally, the pinnacle of, you know, API maturity, not every, not every enterprise wants to monetise their APIs, but eventually, they want to move in that direction, right?

The end of the day, the values with the data you hold and control.

So can you unlock or sort of expose that API, you know, let's say, you know, monetizable model?

Am I okay to export APIs to monetise?

And that's something I see that question gets asked a lot.

I want to enable monetization.

But the real question is, is your API even in a state to monetise?

Right?

Right.

Yeah, that's something that's something worth questioning.

Okay, I was taking notes.

For the sake, for the sake of the conversation, let me rehash what you just said in a summary.

So if we would be looking at maturity of an API programme of API management of our APIs within an organisation, I noted down several different qualities that we would look like, we will look at.

So starting with the way we access those APIs going through, do we have actual an inventory of those APIs and the proper documentation that comes with that inventory?

Then do we have a some sort of a data dictionary?

Do we actually have a semantic understanding of set data that we're exposing through APIs?

And then on top of that, this would be a model dictionary.

So basically, do we have a proper modelling of those APIs, not just exposing raw tables of a relational database, but actually exposing, let's say, that data product we were talking about a little bit earlier.

Then extensibility as a characteristic, can we extend it easily?

Can we actually add to it?

Can we build new functionality with it?

Then with that comes reusability, can we reuse that API in other contexts to create new business, whatever business that may be.

Then deduplication, so the actual effort to reduce sprawl of the same functionality, same data being delivered in a similar way or identical way, but just from different places in the organisation.

And then the actual usability, so how easy it is to use that API, how intuitive it is to be used, because some of those are really not intuitive.

I've seen many of those like that.

And then on top of that, security of set APIs, and then the review cycles, and how often do we come back to actually check on those APIs?

Are they still useful?

Are they still consumed?

Are they still secure?

Do they still have value?

And then that constitutes to, let's say, five levels of maturity.

So first level being functional maturity, well, it works.

Second level being internal standardisation, so we have it according to our internal standards.

We started standardising our APIs so they're actually readable for the majority of the company, let's say.

Then we apply automated governance as third level.

Then we could go to fourth level being the industry standardisation.

So we're actually looking at what the industry is doing, either through PSD, TM forum, healthcare standards, whatever the industry may be.

And lastly, which is a level not everybody will want to go to, would be the monetization of your APIs and monetization of the data products.

Is that a good summary?

Yeah, you nailed it, Carol.

Nice.

What I would also include is the data compliance aspect.

Data compliance.

Yeah.

All right.

Yeah, sure.

Notice.

But you see, now having this, we have the terms in which the access in which we could define what means access, what means inventory, what means a data dictionary in a context of a specific company, what means reusability, what means extensibility.

And now we have more of a frame in which we can have a conversation within companies.

What is our maturity and what do we understand of our maturity based on those qualities that we here have?

And that might be then a very nice and structured conversation to be had.

And even if somebody would just go in and have a chat with a company offering an API management solution, well, we can take those qualities and discuss how they can help us achieve these kinds of qualities, how the tool will help the organisation build on those qualities.

And of course, the tool will not solve all the problems because most of those are organisational problems, not technology problems.

But this is at least from that perspective, we can actually have the conversation and see where we can align and help each other, right?

Which is a good thing.

All right.

Continuing on.

Yeah.

And yeah, this is a really important topic, right?

When we talk to API management, an enterprise, the first thought that an enterprise always has is, yeah, we have an API management, we have an API gateway in place.

So it's always a common sort of a, I would say a convention by now to sort of correlate API management and an API gateway.

And is it the fault of an enterprise?

Not really, because gateways have, I would call them the old, they've been around for quite a few years now, right?

Like 10 plus years.

And it's no sense that no reason why enterprises can't sort of call that as their API management strategy.

But what I've seen is enterprises are increasingly becoming more aware of the problems within their governance gaps.

They're saying that, hey, now that I've realised that a gateway is excellent at the runtime, but as soon as I start imposing expectations of what a gateway should help me to solve, let's say, why can't my gateway secure my APIs?

Why can't my gateways help me design and build them in a consistent way?

So that's when I've seen the limitations, right?

Gateways were never built for sort of an, I would say, lifecycle management.

Rather, gateways were primarily designed and built for traffic management.

They're great at the runtime.

And that's where we see it as a sort of a mismatch in terms of expectations of what a gateway does and what an API strategy should be within an enterprise.

A gateway cannot be sufficient in order for you to manage your APIs.

And that's exactly why we see many enterprises, they adopt an API gateway, and then they start running into challenges, right?

Now, in order to solve for the sort of lifecycle, I need to bring in that aspect to solve for my design, for my testing, for my observability.

So this is something that we see a lot across the enterprises.

Adeeb, can I ask you one thing?

Can you put that phone on do not disturb or a flight mode?

Because I do hear it buzzing in our audience as well.

So that would be nice to just get it out of the way so we can continue.

All right.

Yeah.

So yes, you're definitely right that API gateways, as they were, API gateways is basically a pretty old concept.

It's definitely over 10 years old.

I don't remember when I had the first encounter with an API gateway.

I was already an architect.

So let's say that would be somewhere between 2014 and 2018, right?

So that's a pretty old concept.

And you're absolutely right that gateways, from that perspective, at least back in the day, they were primarily traffic management or security, in essence.

And the difference is, looking at it now and then, we didn't have such a distributed environment.

We didn't have so many clouds and apps living all over the place, which basically gateways at that time were your perimeter defence.

They were placed in DMZs of companies on the edge between the internet and the internal network of a company.

It's changed quite a lot because nobody now uses a DMZ term ever in terms of security of their network because the DMZ no longer exists because we're distributed among several different networks.

If you deploy in several clouds, you're basically managing the connections and secure VPNs between several different networks and you have your applications living pretty much everywhere around the world in that sense.

So that changes the expectations of our gateway in a sense.

Of course, we're dropping that to several other things, but that would be it.

Yeah, that's right, Carol.

And we always ask ourself, with the current sort of limitations or my current setup in place, can we actually do better?

How do I sort of optimise my current setup or how do I sort of get into that federated API management or managing my APIs in a more better, more governed, more secure way?

And that's where we come in with, you know, if you have to view our management as a sort of like a systematic practise, you need to be sort of like a product-centric approach, right?

That means you have that cross visibility.

Everyone has to be on the same page of what is it that you're trying to solve?

What are you building or where are you getting the data from?

Is it from a common repository?

And one thing that's always, I feel a bit bad this many times within enterprises, the developer, no one takes into account the experience of the developer, right?

Because even if it's an API that you expose in your portal, your consumer consuming it is going to be the development team from the other sort of company or your customer.

So people always sort of put an afterthought to developer experience or rather developer enablement and experience.

So how can you actually improve the lives of your developers?

Is it a process that can make it more streamlined or tell the developers, hey, no matter what you design in your API, I'll make sure that your documentation is automatically updated for you.

So you don't need to sort of manually go and update the portal again.

So can you give some level of automation, some level of sort of help towards the developer side of things?

So that's something that I really feel needs to take a bit more importance than what it is today.

Yeah.

And also in terms of the overall application architecture, I see that we see that teams are sort of geo-distributed working in silos.

So that has to slightly change or shift.

People need to start collaborating more, maybe accessing a central repository or a catalogue of your APIs and the data models.

And then even when it comes to sort of like a lifecycle management type of things, right?

Do you have a sort of a predetermined flow of how your API needs to be taken from, let's say, design to deprecation?

Is it a well-defined process?

Or are teams just sort of going crazy and creating their own rules, which sometimes doesn't work, particularly when you're working on larger organisations, it has to be some type of structure.

And finally, on the compliance side of things, it's good to say that you're, let's say you're compliant.

Hey, all my data is, you know, I'm HIPAA compliant, I'm PCI DSS compliant.

I encrypt my data, I store it.

But end of the day, what I see is, let's say you're a third party is using your data or your functionality.

It's sufficient for the third party to get hacked, for your data to get compromised, right?

Because even the third party using your data, your reputation lies on even the third party integrators.

So that's something that I see needs to be taken on priority as well.

And also about the data compliance, because it's not just enough that I have a WAF running or I have a couple of security rules that I enforce to my APIs.

But in the last thousand calls, let's say for getTransaction, was it actually compliant against my schema?

Did my consumers call it with an extra header or a payload that doesn't match my approved schema?

Those type of behavioural analytics, as we call it, is that something that enterprises have access to today?

The answer is no.

But can they get access to that?

It's pretty complicated, right?

The data scattered across, I've seen enterprises have DataDog and Splunk and Elastic and Kibana, New Relic.

So in order to stitch that data together, it's going to be a nightmare of a process.

Even to debug any issue in production, I see teams logging to DataDog, getting the span ID, then going to Grafana, getting the application logs.

And then there's a whole debate for six to seven hours trying to figure out, hey, which application team built this?

This log belongs to which application?

And finally, by the time you decide whether it's an application issue or a system issue or a network issue, you've already breached the SLA.

You're entitled to pay, you're liable to pay a fee.

But this is not a problem that we can fix with API managements and federation itself.

It's a problem of observability in addressing proper metadata in communication, which is where API management or API management federation can help because then we can manage the metadata of communication and standardise it.

So if our users are using the wrong metadata and sending the wrong, let's say HTTP headers, we can catch that and fix that.

So if a user is not sending, consumer is not sending correlation ID on a layer of an API management, we can generate that correlation ID and push it forward to whoever is having that API behind the proxy so that we can still have that observability, at least from our perspective, our end.

Yep.

It has to be centrally accessible, managed, and all the teams have to be in the same page.

Let's just imagine if you tell an SRE, because one of the biggest problem with SRE teams today is they don't have enough context.

They have enough tools.

There's a surplus of tools.

They have different ways in which they can access application logs, but they don't have that context that correlates, hey, this log belongs to this application.

The last change that went into this application was this, and it's a conflict change that brought your API down.

So they don't have enough data to correlate which owner owns this particular piece of information or which application generated this log.

That's something that's missing.

Just imagine if an SRE could have that initial context information available to him.

That means we're talking about bringing down sort of like mean time to resolutions from, let's say, a couple of hours all the way down to one or two hours, all the way down from eight hours to two hours.

That makes a massive difference, even if you shave off a couple of hours.

So that's something that can only be bought through a systematic change.

That's something that teams have to work on.

All right.

But one thing we missed so far, which is exactly addressing the definition of the topic at hand.

So then in that case, with all those problems, what then is federated API management?

Yes.

So federated API management is having that central sort of author who sort of creates those standards, rules, regulations, but you're able to enforce it in a distributed fashion, right?

You have different hybrid clouds, as you mentioned.

Today, it's pretty common to have a mix of GCP, AWS, and Azure across multiple clouds, multiple environments.

I've seen a lot of banks, they try to keep their core operations as an on-prem within their on-prem set of servers, but something maybe not so critical, like just like a, let's say a record maintenance or a public record display, that's something that they expose on the cloud.

That's fine.

But the core fund transfer transactions, processing, those are all held on-prem.

So can I manage the rules that govern how I design, build, test, deploy those APIs?

Can I centrally sort of disseminate that information to have a central sort of catalogue of visibility?

At the same time to have the systems sort of standards and protocols that will govern how my teams, no matter where they're located, are able to build, operate, and govern that API.

So federation deals with the entire process of delegation, decentralised management, at the same time centralised standards and operating procedures.

Okay.

So now contextualising this to tech and looking at your own product, which is API Wiz.

So if we're talking about federated API management, in that case, if we place API Wiz, correct me if I don't understand it correctly, but let me try and contextualise this.

So in that sense, API Wiz serves as the management layer without the gateway itself to manage,

to connect and manage other API gateways and be able to reconfigure them and manage them

centrally so that we can, for example, move away one API definition and API proxy from one gateway

and move it to another, or to set up policies from a single plane of control rather than having

to go to each separate gateway in our organisation.

Do I catch it correctly then?

Yeah.

So API Wiz has connectors and integrations to every data plane you can imagine, like all your Apigee, Kong, or any other gateway.

But at the same time, we make sure that you can keep your data plane fluid.

So whether you're using Apigee today or Kong tomorrow, it doesn't matter to us.

The way in which you design, the way in which you build, test, deploy, observe, monetise, that stays consistent irrespective of which data plane you're using today.

Because pretty much these have become a commodity today, right?

Due to cost or performance reasons, it's not uncommon for an enterprise to say, hey, Apigee is too expensive, let's move to Kong.

Or let's say, Kong's not working out, let's take up Tyke.

So these decisions happen on the fly, pretty common within enterprises, right?

But what happens to all the effort that developers did?

They spent several hours, several months, sort of configuring Apigee.

They tied down their API management practises to a data plane, which is, we feel that it's wrong, right?

Need to sort of, I would say, decouple your API strategy from your gateway, right?

The gateway is purely a runtime component.

So it's great testing your APIs and running your proxies, but never tightly couple your API strategy to the gateway, right?

Never couple your business logic, never couple your API strategy on the data plane.

That can change anytime, and it's going to harm you in the long run.

So that's what I would summarise.

Okay.

So basically, if somebody would want to start federating API management, they basically would need to create a centralised control plane from which they could actually govern all those various API gateways and be able to move their definitions of proxies throughout their ecosystem of those products.

And then decommission whole data planes at hand from a specific vendor in favour of another, et cetera, et cetera, et cetera.

So be able to reconfigure on the fly with all the definitions held centrally, managed centrally, and then deployed as implementations for specific gateways in a standardised manner.

Correct.

You don't need to reconfigure your APIs every time it's changing from one data plane to another, because the artefacts are already centrally stored on the control plane.

So you don't need to change the process.

You just need to change a connector to your API gateway.

That's it.

Which is not a very common thing with API management tools, that they have that.

Of course, they have that within the scope of their own tools, so that if you have multiple gateways of a single vendor, you do have that control plane to those gateways, but not every time to other gateways.

I've seen already some API managements that have API gateways actually do that federation so that they connect to other API gateways to manage them as well.

But I've never seen that on a scale of every possible solution ever, only of just a few, the most common ones, but that's just an add-on functionality to that product rather than the product itself.

Correct.

All right.

All right.

That's interesting.

I think for a lot of people, that would be a new topic to look at it this way, to decouple, well, not decouple, but loosen the coupling between the control plane and the actual implementation of a gateway.

I don't think that's common yet, but given the distribution of our

systems, applications, and how we deploy everywhere based on what is the expertise of a team or what

is the current need, I think that's going to be more and more of a common use case in the

coming years, especially now with further sprawl of APIs around agentic AI, et cetera,

that's going to create even more chaos that would have to be centrally managed.

That's right.

There's going to be a lot more burst in the number of not just the volume of data, but also today I see pretty much startups being created overnight, right?

On the fly.

The new AI has made it possible to generate a website in seconds, build functionality in a matter of seconds.

So that's something that I see it's going to propagate a lot more in the coming years.

Yeah.

That's definitely a game changer in terms of speed of creation and then also dying of these kinds of companies and SaaS solutions that they just pop up, several of them, then they die off and they pop up again.

And it's just an ongoing cycle that's sped up drastically.

Normally it took a few years for one company to establish and then to vanish if it doesn't come to fruition.

But now it's just like they pop up and they just die, right?

So it's a lot faster feedback loop in that sense, but that requires better tools to control those APIs.

All right, Adeeb, I know in that sense you're quite busy, so we're going to have to cut short.

I know you have other things planned for tonight and that's what it is.

That's a life of a CTO, I suppose, being busy, busy, busy.

So summing up, we've talked about what an API management is.

We talked about what are the different components and what are different governance issues, the knowledge divide, governance divide, and the sprawl.

Most of this being the organisational issues rather than technical issues, but we can actually use pieces of technology like API Waze to help us at least try and govern things and standardise them and bring our API maturity to a manageable level.

But that comes always with an organisational change.

Without that organisational change, no tool will help in that sense.

All right, given that, I'll switch for a second to a different slide set.

So if you're new here, please go read more about what we're doing at Bridging the Gap and the articles we do on application integration topics at Bridging the Gap.

You can follow us on Substack as well and subscribe to the YouTube channel if you haven't done so before.

And the next live stream over is understanding cognitive flow.

So we're jumping back to psychology a little bit and psychology within different IT teams and whatnot.

This is going to be quite a challenging and interesting topic because not a lot of us actually understand psychology in IT and that's going to be a fun conversation.

We already talked about this some time ago on the live stream.

We're now diving deeper into more of an indicator understanding that cognitive load.

So that's next week, next Wednesday.

Feel free to join on that call, that live stream.

And that said, we'll be closing our stream.

So Adeep, thank you so much for joining today and explaining the lovely topic of federated API management.

I think that was a very, very condescending pill of knowledge there.

Because, well, we could probably talk for an hour more, but time is time.

So thank you for joining.

Thank you for having me.

It was really cool to have you here.

And hopefully we'll pop on another live stream someday, talk some more about federation of API management and how we can do that in a little bit more detail and showcase some more live examples of API federation and different cases that you might have stumbled upon with customers.

That would be also a very cool thing to see someday.

Sure.

Look forward to that.

All right.

That said, thank you all for your attention.

Thank you all for being with us.

For those who haven't been with us live, I hope you liked the recording and see you some other time.