Welcome!

Apache Authors: Pat Romanski, Liz McMillan, Elizabeth White, Christopher Harrold, Janakiram MSV

Related Topics: Java IoT, Industrial IoT, Microservices Expo, Open Source Cloud, Machine Learning , Apache

Java IoT: Blog Feed Post

Agile Architecture

The platform architecture defines common services that manage business delivery

The English language is well known for its subtlety. Sometimes it’s a delight, but on other occasions it can be very frustrating. If I use the term Gothic Architecture you will immediately understand I am describing a style of architecture that flourished in medieval times. And if like me you are interested in ecclesiastical architecture you will know that this style was used in many of the great cathedrals and churches across Europe, which were distinctive because of key architectural patterns that enabled great increases in height and internal light of the buildings without increasing the size of supporting pillars.

Now if I use the term Agile Architecture, what am I referring to? In today’s Agile world I would hazard a guess that most readers will think I am referring to the architecture techniques and tasks undertaken in the context of an Agile software development project, not the collection of patterns and practices that enable agile business systems. That is, an architecture that enables agility.

This potential for miscommunication is a core issue for enterprises. There is ample evidence that Agile Architecture is a primary contributor to business agility, yet we do not have a well understood architecture management system that integrates with Agile methods.

Let’s use an example readers may be familiar with. Amazon CEO Jeff Bezos famously [1]issued an edict that laid down some key architecture principles to Amazon development teams that I will summarize as:
· All teams will henceforth expose their data and functionality through service interfaces.
· Teams must communicate with each other through these interfaces. There will be no other form of interprocess communication allowed.
· It doesn't matter what technology they use.
· All service interfaces, without exception, must be designed from the ground up to be externalizable.
· No exceptions.

What Bezos did here was to lay down key business and technology architecture principles that you might reasonably conclude were central to the extraordinary level of business agility that we have seen demonstrated by Amazon.com, Inc. That widely circulated edict contained the foundations of the Amazon reference architecture.  

In the October 2004 CBDI Journal[2] we commented, “Two of the most successful and enduring dotcom start-ups, Amazon and eBay, now expose their core applications as Web Services. In doing so they have created a new class of platform that could have a profound impact on end-user organizations and IT vendors alike.”

And so the reference architecture became the enabler of growth and agility for the Amazon business, not we understand[3] as a grand plan, but through natural technological evolution. The services formed the platform that allowed the extraordinary expansion of the Amazon business that I would be certain not even Jeff Bezos imagined, back then in 2004. That is real business agility, and it was delivered by smart architecture backed up by clear policies and realized by agile processes.

Although Amazon has clearly evolved in pursuit of solutions to specific business opportunities and challenges, it’s also clear they have established a de facto architecture and architecture management system that guides the work of the many product delivery teams and ensures consistency of approach where it’s required. Let’s consider how an enterprise might establish a similar agile architecture management system.

A reference architecture articulates primary principles that are typically central to an entire enterprise. Principles should be focused on establishing the product and solution independent environment in which agility can be delivered and maintained, so they would be stable over time. We might refer to reference architecture as a Level 1 architecture perspective (L1) that exists purely as a set of models and guidelines.

Larger enterprises should explore the business value potential of platform based architecture as a mechanism to deliver cross enterprise consistency of core reference architecture behaviors and to enable closer integration with the wider ecosystem including customers, suppliers, end consumers etc. This is an extended management services platform which encapsulates the technology infrastructure and enables rapid delivery of business services.

The platform architecture defines common services that manage business delivery including security, life cycle management, change management, release management and operations, as well as catalogs, eCommerce, B2B, regulatory control and risk management, standardizing these key capabilities and reducing the footprint of business domain services. The platform will also manage important behaviors that deliver on specific business goals such as scalability and availability. For example, Amazon services are usually very fine grained, specifically to reduce the scope of each service in order to facilitate narrow focus SLAs and maximize scalability by reducing individual service complexity. We might refer to platform architecture as a Level 2 architecture perspective, engineered to be relatively stable in support of  large numbers of business services and consumers, but also engineered to evolve and respond rapidly to business and technology change. Not all enterprises will see business value in making their platform and business services available to their ecosystem, but some will.

Enterprises clearly vary considerably in their make up in terms of geographic and organizational, product and process standardization and differentiation, but typically there will be considerable potential for an inventory of shared assets that leverage agile architecture to support business agility. The assets may include:
· Common services, frameworks and components that are designed to deliver common behaviors to all parts of the enterprise. For example core services that establish genuinely enterprise wide services such as Customer, Ticket, eCommerce etc; services that deliver business value by standardizing common business services and processes.

· Configurable services, frameworks and components that are designed to provide common behaviors but are engineered to be customizable in local situations to accommodate many aspects of localization ranging from the simple – taxation, geography etc, to the complex – variant ordering patterns, variations in event and process sequence dictated by local de facto business practices. Configurable services may provide business value simply by providing reusable components, or they may establish a common core of business process and information that establishes common reporting and regulatory control in a local context, or both. Configurable services may also be an important time to market strategy for service providers who customize their services for each client or customer group.

· Information architecture and services. Establishing a coherent approach to information is commonly a major issue for large enterprises and this architecture level defines an integrated approach for structured and unstructured (big) data, transactional and reference, enterprise reporting and regulatory control and so on.

Common and Configurable assets together with the Information Architecture might form a Level 3 architecture perspective and be widely applicable across a large, distributed enterprise.  

We then have two further levels which are closely related, Family Architecture and Product Line Architecture. Whilst many architects chose to view Family and Product Line as synonyms, I recommend that they are kept separate. A Family architecture is a domain framework that is much more specialized that L3 assets that would be applicable on a broader basis. The Family architecture establishes core business (domain) services and possibly other artifacts specific to the domain, where the domain is likely to be a subject area or a cluster of major types. For example Customer, Supply Chain, Manufacturing, Risk etc. Families are also commonly acquired products.

In contrast Product Line architecture is what it says – it’s the architecture for a product offering. The product is an offering that has direct relationship to end customer revenue and usually continuity of purpose over multiple releases. Although from a narrow technical perspective the Product and Family architectures might be similar, the way a product is managed must mirror the business product life cycle. Family architectures may therefore be engineered for stability, whereas, depending on the industry sector, product line architectures may be engineered for maximum agility and minimum response time.  

Finally we have the Solution architecture level, the architecture specific to solution project delivery, where the focus is on feature architecture and integrating solution architecture with the Level 1 to 5 architecture perspectives. It’s important to note that where product line architecture is used, then this may subsume the Solution architecture.

These six architecture levels provide us with a nomenclature for agile architecture that will be central to managing agility into the delivered product/solution. The architecture perspective guides the structure of programs and projects and the incorporation of architecture and reuse goals into delivery charters. The architecture also provides traceability and governance over realization of core architecture principles.

The question of how Agile Architecture integrates with Agile delivery is likely to prove contentious because architecture introduces a form of direction that contradicts Agile concepts. Yet the lessons from Amazon are insightful. The most senior business management need to be fully engaged and actively leading the development of architectural direction. Further in large enterprises customer project demand needs to be managed and aligned with business strategy and architectural direction.

There’s no reason why these Demand and Definition processes shouldn’t adopt Agile concepts, notably cross functional teams, time boxes and backlogs. The outcomes should be excellent visibility and traceability of key strategies and policies that provide real clarity of purpose for projects, that will increase the probability of success. In a typical large enterprise use of existing (or well understood) organizational concepts, adjusted to use aspects of Agile methods as discussed, will meet less organizational resistance. For example:  

1. Architecture Review Board (ARB) or equivalent, a cross functional team (senior representatives of business, product management, architecture and delivery), that provide direction and funding to all architecture development.
2. Design Authority (DA), also a cross functional team (domain specific expert level representatives of business, product management, architecture and delivery), that transform raw customer demand stream into project charters and manage the portfolio view. It is the DA that takes responsibility for aggregating and decomposing customer and strategic demand, chartering Common, Product Line and Family architecture, typically as integral elements of delivery projects, which can demonstrate business value.
3. Investigatory architecture projects – short duration projects that validate assumptions prior to chartering composite architecture/delivery projects. Sometimes carried out as part of a Definition Phase activity concurrent with outline requirements and knowledge discovery. Using patterns as a mechanism to increase consistency of architecture decisions and communicate them to delivery projects at sensible level of detail that is useful to delivery teams.  Recommend includes delivery team members as appropriate.
Note this is a recursive model, and the process may executed at enterprise and program level.

You may ask where Enterprise Architecture is in this. The answer is that enterprise architecture is a role and responsibility that must coordinate and govern all levels of architecture. Enterprise Architects are most likely to be assigned to a specific architecture perspective level. The notion of, “one architecture to rule them all” really doesn’t exist.
Each enterprise should develop its own architecture management approach, and integrate this into an end to end architecture, delivery and governance process. The term Agile Architecture should be used to describe and deliver architecture that facilitates the agile business by compliance with reference, platform and other architectures that facilitate evolution, customization and plug and play. Faster cycle time and quality outcomes are then a function of both the reusable patterns and parts available for assembly and the Agile delivery process.  

In medieval times the builders of the Gothic cathedrals didn’t start their designs from scratch. But equally they didn’t have finely detailed (ivory tower) plans – the technology didn’t exist to support that. Master builders moved from city to city bringing their proven architecture in their heads, often together with experienced craftsmen, to new projects. Craftsmen and master builders together tried out new designs and gradually evolved core patterns such as the flying buttress, which became standard components in cathedrals across Europe. Sometimes the great buildings fell down during construction and the builders had to adapt the architecture and try again. They were truly early adopters of Agile methods as they combined architecture and build in what clearly was from time to time an empirical delivery approach, but they also had their equivalent of a reference architecture and patterns that enabled systematic reuse of proven designs. Of course their delivery cycle time was a little longer than today’s Agile project!


Talk to Everware-CBDIabout the Agile Enterprise Workshop. This is currently available as an in-house, intensive workshop. Public scheduled classes will hopefully follow next year.




[1] Amazon and eBay Web Services, The New Enterprise Applications? By Lawrence Wilkes, CBDI Journal October 2004


[2] Inadvertently published by Steve Yegge, 2011, in a comparison of Google and Amazon practices. http://upalc.com/google-amazon.php

[3] Werner Vogels, 2006, SOA creates order out of chaos @ Amazon, Rich Seeley, Search SOA

Read the original blog entry...

More Stories By David Sprott

David Sprott is a consultant, researcher and educator specializing in service oriented architecture, application modernization and cloud computing. Since 1997 David founded and led the well known think tank CBDI Forum providing unique research and guidance around loose coupled architecture, technologies and practices to F5000 companies and governments worldwide. As CEO of Everware-CBDI International a UK based corporation, he directs the global research and international consulting operations of the leading independent advisors on Service Oriented Application Modernization.

@ThingsExpo Stories
SYS-CON Events announced today that Secure Channels, a cybersecurity firm, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Secure Channels, Inc. offers several products and solutions to its many clients, helping them protect critical data from being compromised and access to computer networks from the unauthorized. The company develops comprehensive data encryption security strategie...
In his session at @ThingsExpo, Sudarshan Krishnamurthi, a Senior Manager, Business Strategy, at Cisco Systems, discussed how IT and operational technology (OT) work together, as opposed to being in separate siloes as once was traditional. Attendees learned how to fully leverage the power of IoT in their organization by bringing the two sides together and bridging the communication gap. He also looked at what good leadership must entail in order to accomplish this, and how IT managers can be the ...
Recently, WebRTC has a lot of eyes from market. The use cases of WebRTC are expanding - video chat, online education, online health care etc. Not only for human-to-human communication, but also IoT use cases such as machine to human use cases can be seen recently. One of the typical use-case is remote camera monitoring. With WebRTC, people can have interoperability and flexibility for deploying monitoring service. However, the benefit of WebRTC for IoT is not only its convenience and interopera...
There is only one world-class Cloud event on earth, and that is Cloud Expo – which returns to Silicon Valley for the 21st Cloud Expo at the Santa Clara Convention Center, October 31 - November 2, 2017. Every Global 2000 enterprise in the world is now integrating cloud computing in some form into its IT development and operations. Midsize and small businesses are also migrating to the cloud in increasing numbers. Companies are each developing their unique mix of cloud technologies and service...
When shopping for a new data processing platform for IoT solutions, many development teams want to be able to test-drive options before making a choice. Yet when evaluating an IoT solution, it’s simply not feasible to do so at scale with physical devices. Building a sensor simulator is the next best choice; however, generating a realistic simulation at very high TPS with ease of configurability is a formidable challenge. When dealing with multiple application or transport protocols, you would be...
SYS-CON Events announced today that App2Cloud will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct. 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. App2Cloud is an online Platform, specializing in migrating legacy applications to any Cloud Providers (AWS, Azure, Google Cloud).
IoT is at the core or many Digital Transformation initiatives with the goal of re-inventing a company's business model. We all agree that collecting relevant IoT data will result in massive amounts of data needing to be stored. However, with the rapid development of IoT devices and ongoing business model transformation, we are not able to predict the volume and growth of IoT data. And with the lack of IoT history, traditional methods of IT and infrastructure planning based on the past do not app...
To get the most out of their data, successful companies are not focusing on queries and data lakes, they are actively integrating analytics into their operations with a data-first application development approach. Real-time adjustments to improve revenues, reduce costs, or mitigate risk rely on applications that minimize latency on a variety of data sources. Jack Norris reviews best practices to show how companies develop, deploy, and dynamically update these applications and how this data-first...
Intelligent Automation is now one of the key business imperatives for CIOs and CISOs impacting all areas of business today. In his session at 21st Cloud Expo, Brian Boeggeman, VP Alliances & Partnerships at Ayehu, will talk about how business value is created and delivered through intelligent automation to today’s enterprises. The open ecosystem platform approach toward Intelligent Automation that Ayehu delivers to the market is core to enabling the creation of the self-driving enterprise.
Internet-of-Things discussions can end up either going down the consumer gadget rabbit hole or focused on the sort of data logging that industrial manufacturers have been doing forever. However, in fact, companies today are already using IoT data both to optimize their operational technology and to improve the experience of customer interactions in novel ways. In his session at @ThingsExpo, Gordon Haff, Red Hat Technology Evangelist, shared examples from a wide range of industries – including en...
Consumers increasingly expect their electronic "things" to be connected to smart phones, tablets and the Internet. When that thing happens to be a medical device, the risks and benefits of connectivity must be carefully weighed. Once the decision is made that connecting the device is beneficial, medical device manufacturers must design their products to maintain patient safety and prevent compromised personal health information in the face of cybersecurity threats. In his session at @ThingsExpo...
"We're a cybersecurity firm that specializes in engineering security solutions both at the software and hardware level. Security cannot be an after-the-fact afterthought, which is what it's become," stated Richard Blech, Chief Executive Officer at Secure Channels, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
SYS-CON Events announced today that Massive Networks will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Massive Networks mission is simple. To help your business operate seamlessly with fast, reliable, and secure internet and network solutions. Improve your customer's experience with outstanding connections to your cloud.
SYS-CON Events announced today that Grape Up will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct. 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Grape Up is a software company specializing in cloud native application development and professional services related to Cloud Foundry PaaS. With five expert teams that operate in various sectors of the market across the U.S. and Europe, Grape Up works with a variety of customers from emergi...
Detecting internal user threats in the Big Data eco-system is challenging and cumbersome. Many organizations monitor internal usage of the Big Data eco-system using a set of alerts. This is not a scalable process given the increase in the number of alerts with the accelerating growth in data volume and user base. Organizations are increasingly leveraging machine learning to monitor only those data elements that are sensitive and critical, autonomously establish monitoring policies, and to detect...
Everything run by electricity will eventually be connected to the Internet. Get ahead of the Internet of Things revolution and join Akvelon expert and IoT industry leader, Sergey Grebnov, in his session at @ThingsExpo, for an educational dive into the world of managing your home, workplace and all the devices they contain with the power of machine-based AI and intelligent Bot services for a completely streamlined experience.
Because IoT devices are deployed in mission-critical environments more than ever before, it’s increasingly imperative they be truly smart. IoT sensors simply stockpiling data isn’t useful. IoT must be artificially and naturally intelligent in order to provide more value In his session at @ThingsExpo, John Crupi, Vice President and Engineering System Architect at Greenwave Systems, will discuss how IoT artificial intelligence (AI) can be carried out via edge analytics and machine learning techn...
When shopping for a new data processing platform for IoT solutions, many development teams want to be able to test-drive options before making a choice. Yet when evaluating an IoT solution, it’s simply not feasible to do so at scale with physical devices. Building a sensor simulator is the next best choice; however, generating a realistic simulation at very high TPS with ease of configurability is a formidable challenge. When dealing with multiple application or transport protocols, you would be...
With tough new regulations coming to Europe on data privacy in May 2018, Calligo will explain why in reality the effect is global and transforms how you consider critical data. EU GDPR fundamentally rewrites the rules for cloud, Big Data and IoT. In his session at 21st Cloud Expo, Adam Ryan, Vice President and General Manager EMEA at Calligo, will examine the regulations and provide insight on how it affects technology, challenges the established rules and will usher in new levels of diligence a...
An increasing number of companies are creating products that combine data with analytical capabilities. Running interactive queries on Big Data requires complex architectures to store and query data effectively, typically involving data streams, an choosing efficient file format/database and multiple independent systems that are tied together through custom-engineered pipelines. In his session at @BigDataExpo at @ThingsExpo, Tomer Levi, a senior software engineer at Intel’s Advanced Analytics ...