We definitely do not want to do “quick & dirty” Architecture, but still, we might just not have the luxury (time & resources) to do “clean” Architecture either. So, what do we do?
I’ll put it simple – what we need is – build quick, but sustainable Architecture.
I guess, it’s about finding the right measure of pragmatism (Lean EA) vs. discipline (framework driven EA). And sometimes, it is quite individual – because we are Architects; we might not always build the solutions in the same way; and this is okay as long as the problem is solved, right?
Integration (Enterprise) Architecture
Integration Architecture as a concept has started with Point-2-Point integrations of various Applications. However, business needs were dictating the evolution into Middleware “orchestrated” integration of various Services exposed or consumed by connected Systems.
Now, Service Oriented Architecture (SOA) is not only about Middleware. While Middleware is usually an integral part of the SOA; evolution to SOA is more in “philosophy” of the Integration Architecture, than in the actual ways how Applications or Systems communicate (Point-2-Point or via Middleware). In fact, there would always be some Point-2-Point integrations, even in the most “modern” SOA landscapes – we may call it Legacy Application Interfaces(s), but in reality, it won’t be Legacy only...
What is this SOA evolution about?
I will summarize it in few key SOA concepts:
Service Contracts (incl. standard integration, documentation etc.)
Litmus “Test” Standard (incl. business aligned, composable, external description, reusable, technically feasible)
I am not listing Middleware as a mandatory "tool" for all interfaces, but it is, of course, expected for the majority of the Integration Services. Having a unified view, through Middleware, would further provide easier management of the Integration Service Lifecycle – from the design, implementation, operation, and finally to its replacement & retirement.
Also, worth noting, SOA is not limited only to On-prem or only to On-cloud Architecture – there is no such limitation.
Still… Is SOA enough for the new & modern Intelligent Enterprise? Is SOA sufficient to deal with ever increasing appetites for the Business Process Integrations, UI Integrations, "Things" (IoT) & AI/ML Integrations etc.?
I would say – no, we do need more…
What is Intelligent Enterprise?
While there are many definitions for every IT “buzzword” I am not going to quote anything – I will share my interpretation – what Intelligent Enterprise is.
Yes, sure, Intelligent Enterprise works with “smart” things like AI/ML, IoT and integrates it into its Business Processes. But this is just tip-of-the-iceberg… Why do we need so many “smart” things?
Because our business must be fast, it needs to act (not only react), to disrupt (not only adapt)… Intelligent Enterprise is “smart” with many integrated services, internal or external; it is a part of the complex Eco-space, connected with other (Intelligent) Enterprises. Thus, Intelligent Enterprise requires more than integrated Applications and Services – it requires an environment supporting Interoperability.
Integration Architecture for the Intelligent Enterprise
In the Serverless integration, are we going to run Microservices or Webhooks through Middleware? Probably not – does not make much sense.
Shell we under all circumstances rule-out IDoc or RFC integrations in favor of SOAP and OData APIs? No, why should we do that? If something works and provides necessary business value, don’t touch it.
Do we use Legacy & Propriate protocols (e.g. for IDoc, BDoc, RFC etc.) for Direct(ware) Point-2-Point connections or do we use Middleware as well (when supported)? In fact, we might apply either of the approaches – depending on the specific integration use-case.
Shell we be open to provide content “information” without integrating content itself – e.g. by using iFrame? Sometimes it may be slow, but could be very useful in specific use-cases.
In the Interoperability Integration Architecture, I would not try to “visualize” integration flows – I would rather focus on the 3 layers:
Core Enablement – it’s not a “tool” it’s an envelope of Middleware approach (there could be more than one Middleware), Direct(ware) approach and Serverless approach. Core Enablement provides both, “tools” and “guidelines”, to facilitate integration flows.
Integration Capability – represents a collection of all interface technologies; APIs, Events, Microservices, Legacy & Propriate interfaces etc. providing integration either through the orchestration platform or directly.
Interoperability is the layer where Business Systems or Services are exposing and consuming required Business Object (data and its representation).
Why am I separating Serverless approach from other approaches?
I find (but that’s my view) Serverless integration to be a “synonym” for a lightweight integration – perhaps too light to be consumed via Middleware, but still may be very much reusable, and even orchestrated in its environment (e.g. it’s used for Cloud native apps – and obviously can be monitored and orchestrated within the Cloud instances).
Why Direct(ware)? Why not simply saying Point-2-Point?
Well, Point-2-Point would somehow imply I am talking about Legacy & Proprietary interfaces (e.g. RFC, BDoc) – and my view on “Directware” integration is a bit wider. Yes, Point-2-Point would be direct integration (although RFC could also go through Middleware), but what about iFrame? Isn’t this (usually?) direct integration from the website page to another web page? And still, I would not call it “Legacy & Proprietary” interface. I guess, I just felt like putting some buzzword like “Directware” which has wider meaning (at least for me)...
And of course, if something is used in the direct integration between two Systems, it does not mean this cannot be re-used in some other integrations (e.g. same RFC end point can be called by several Systems, same “embedded HTML” as an iFrame can be used by multiple websites etc.). So, Direct(ware) does not mean this is one-time build integration…
Key principles for the “new” Integration Architecture approach
When designing “integration” do not let API mindset limit the design. We must bring business value – meeting current needs and provide a solid foundation for the future needs (those we can foresee). And still, we must be able to adapt to changes (changed business requirements/needs) when they happen – and they will happen…
I see these key guiding principles:
Consumer driven integration – meet the business requirements, do not “box” (and fit) integration to API constraints – business needs are driving for the Interoperability;
Events, not replication – we do not need to store same data over and over again;
Reusability – we might have something we can re-use, keep documentation, keep re-using whenever possible
Keep it simple – if we have standard API (provided by the Vendor), let’s try to use it;
And still, we are usually not able to keep up with these principles… It is amazing how often we are “building” new, when something existing is already “sitting” and waiting to be re-used… Also, It is amazing how often we are “forced” to architect data replications of large data sets from one Transactional System to other Transactional Systems. Do we really need to replicate “zillions” of records from one System to another? Are the “consumers” in the other system actually going to work simultaneously on all those records, all its attributes, all the time? Or they just need to access some attributes, at the given time? Can’t we just implement GET method to read some attributes from the requested records when they are actually needed… Let’s be honest, in most of the cases this GET would do the trick (and it would be cheaper than all unnecessary traffic and storage used for replication)… And if “other” System is also subscribed to Event PUSH from the source System (notifying it of any changes) as well, then we can probably cover all possible needs of the Interoperable Transactional Systems.
But we are often constrained by our old habits…
In the Intelligent Enterprise, we cannot afford to have API driven Integration Architecture. There is no single Middleware, there is no single style or pattern for building the Integration Services.
We should be “smart” and “open” (for new things, new styles, and new patterns), when building the Integration Architecture. I guess, I can say, we simply have to be agile enough when building an Interoperability Environment in the “modern” Intelligent Enterprise.
After all, Agile Enterprise Architecture is about interoperability, not only about integration…