Intelligent Enterprise

I saw videos on Youtube addressing “Signs of intelligence” in people - one good one was from Success Formulas. Contemplating how intelligence translated to action creates value and how that applies to organisations vs people. So, here are 13 marks of intelligence in enterprises:

  1. Curiosity - A desire to be aware of, to know and to understand. This applies to the environment, competition, economy, technology and many other factors. Enhanced by open culture, available resources, sharing and availability of open channels and information

  2. Adaptability and Flexibility - Willing to change procedures, processes, ways of working for the better, especially based on facts and evidence. Supported by investment in research, training and mentoring. Encouraged by value driven culture

  3. Sense of Humour - Being able to find the funny in the absurd or adversity. Being able to see our own faults, recognise them, acknowledge, learn from them and move on

  4. Learning from Mistakes - Not just allowing mistakes, but actively learning from them and spreading the learning so we don’t make them again

  5. Versatile Memory - Recording things, organising things, sharing knowledge, use of ontologies, making knowledge explicit rather than than tacit. Supported by semantic technology, graph database, AI

  6. Emotional Intelligence - Paying attention to the people side, desires, aspirations. Creating a good culture which values individuals and looks to satisfy their needs, but also demands high standards and delivery

  7. Intellectual Humility - Our way is not the only way. We can always learn more. Take in research, see what competitors are doing, find new models. Being open

  8. Creativity - Sparked by open innovation, forums, pet projects supported by enterprise resources, some pure research just based on curiosity. Leaving space for serendipity

  9. Open Mindedness - Accepting of new ideas, diversity (age, race, gender, culture, language, income, values…)

  10. Effective Communication - In all forms. To the ecosystem surrounding us (Partners, Regulators, Industry Bodies, Interest Groups, Unions, Media, Employees, Public, Shareholders…) via various media. Also encouraging free and open communication internally. Creating collateral which clearly communicates who we are and what we are about

  11. Self Awareness - Reflection, good metrics. Knowing our strengths, weaknesses, opportunities and threats. Able to focus on what we do uniquely and well while outsourcing other things or improving them

  12. Strategic Planning - Thinking long term, but acting in the present in alignment with vision. Enhanced by business and enterprise architecture

  13. Range of Interests - Diversification. Not putting all our eggs in one basket / product / service or small group of customers. Being aware of the “adjacent possible” to come up with new, possible Blue Ocean offerings

Here’s to more intelligent enterprises in the coming year, leading to Desireable Futures.

Pareto and saying “No”

In contemplating the New Year and plans for the future, I came across a simple process by Vicky Zhao that looks at 1) Review 2) Plan 3) Prioritise using five techniques a) Pareto b) confirmation bias c) inversion d) "one thing" e) SMART objectives. I was struck by how similar these are to an architecture process and how we can exploit two of these ideas in particular.

The first is Pareto or 80/20 analysis in the review of past performance. Simply put, Pareto analysis tries to find the things that may have consumed 20% of effort or resources, but produce 80% of the desirable results. This is a great way to identify those things we should double down on in the future. If we can spend 60% of our effort and resources on them in future, we may be able to generate 3x the desirable results!

The inversion idea in planning is to start with the end in mind. We do this routinely in architecture through developing a vision or scenarios. We can then decompose this to identify what will be required to make it a reality. We can do this for several time horizons to give us short, medium and long term goals. Think Transition Architectures.

Next we need to ensure that we are not distracted or diverted from the focus we need to progress meaningfully and continuously. The “one thing” in the planning approach encourages finding just one key thing/theme to focus on per time horizon.

Steve Jobs extolled the virtues of focus powerfully:

"People think focus means saying yes to the thing you've got to focus on. But that's not what it means at all. It means saying no to the hundred other good ideas that there are. You have to pick carefully. I'm actually as proud of the things we haven't done as the things we have done. Innovation is saying no to 1,000 things."

Extending the architecture idea of gap analysis, we can look at:

  • Which things are on the 20% Pareto list? We should be saying “yes” to these and “no” to the others

  • What should we Stop doing? Many of the things in the 80% effort Pareto list can fall here

  • What should we Start doing? These would be things that support our goals and vision that are not already in our capabilities

  • What should we Change? This can include improvements in efficacy, quality or efficiency

Finally, we need to further decompose the goals remaining to objectives and ensure they are Specific, Measurable, Achievable, Relevant and Time Bounded (SMART).

Happy planning. Remember Pareto and the power of saying “No”. Also keep in mind this wisdom from James Clear, author of Atomic Habits:

“Goals are good for setting a direction, but systems are best for making progress”

Leveraging Assets

An asset was traditionally something you own which had value or which you could use to derive value. An example of the former would be cash or gold. An example of the latter would be an item of equipment.
We can update this in two important ways:

  1. Assets can be virtual or digital, so we could have something like a skill, a design, a patent or a recording

  2. We don’t necessarily have to own them to derive value from them

Some of the fastest growing and valuable companies do not own the assets they leverage. Uber does not own cars; AirBnB does not own accommodation; YouTube does not own the content it serves.

Virtual assets, such as a design, can be very valuable. We can profit from royalties, copyright, trademarks etc. without necessarily ourselves making the product or delivering the service. Consider the inventor of the crown bottle cap, William Painter, whose company received a royalty on every cap used for several decades!

Digital assets are also profitable. A music track is recorded once, but can be listed on thousands of websites virtually for free, then duplicated, again virtually for free and shipped to consumers, again almost for free. This can occur millions of times, generating substantial revenues while not parting with the original asset.

The best though, is using someone else’s assets to deliver value. Uber, for example, uses the assets of owners (cars), the assets of drivers (skill and time), the global infrastructure of the Internet (funded by advertising, corporates and governments) and the asset of the user (cell phone) to deliver a valuable and desirable service.

In doing business and architecture planning, it is useful to contemplate Asset Leverage.

First list assets. Look for things that you own, things that you know, things that you know how to do. Try to find things in the categories of physical (e.g. property, stock, equipment); monetary (cash, investments, shares, bonds etc.); knowledge/designs/patents (e.g. books, recordings, designs, models); virtual (e.g. skills, customer goodwill) and digital (e.g. recordings, images).

Next think about assets you do not own that you can leverage. Examples include those of Partners (e.g. supplier knowledge, skill, equipment, stock); Customers (e.g. premises, network, computing, cell phone, time); Investors (expertise, connections); Infrastructure (e.g. Internet, public facilities); other Owners of something you need (e.g. Accommodation, Cars, Images, Location data).

Figure out to what extent you are currently leveraging the assets. Look for opportunities to leverage them to a greater extent. A great deal of value can be unlocked this way. You can find the best opportunities by looking for those assets that can generate a lot of value that you can access with relatively little effort or expense.  

#businessarchitecture #strategy #businessanalysis #digitaltransformation #assets

Stumbling towards AGI (Artificial General Intelligence)

Elegant Architecture overcomes limited and messy implementation?

A new article discusses Hugging GPT which uses Chat GPT as a human interface and executive controlling module to control tasks to complete a goal. The tasks are delegated to specialist AI models that perform narrow functions well. The video discusses the ideas and is a great introduction to the paper.

Better Search: Will ChatGPT (or similar) displace Google?

Google has become indispensable in our work and personal lives. Finding products and services, checking out reviews, finding the cheapest supplier and doing professional research. Google has built a $150Bn advertising revenue business on top of that ubiquity.

The ChatGPT large language model from OpenAI burst on the scene recently, attracting over a million users in a week. It boasts a conversational interface accepting complex queries in natural language, a human language response and allows to refine our search, seek more detail or pursue other aspects easily. This style of interaction is extremely attractive - it’s a bit like having a hugely knowledgable human expert on tap to instantly understand our questions and answer in an accessible paragraph or two. It raises the question “Is this the future of search”? Fuel has been added to the fire of this debate with the investment by Microsoft of a further $10Bn in OpenAI. Remember that Microsoft has long promoted Bing in competition with Google search.

But not so fast… The results from ChatGPT are not always accurate. It is based upon a predictive model which has ingested huge amounts of data from the Internet and document sources. Because it is a mathematical model based on probability, it will favour average and mainstream opinions from its training set. It can be prompted to produce factually incorrect answers which are stated very convincingly as facts. Annoying if the recipient already knows the facts, but dangerous or misleading if the recipient does not. The model is also trained on this corpus of data at a given time, in a “batch” mode. So it may not reflect information recently published, or which has been updated since the last training cycle.

OpenAI and others wanting to promote these kinds of systems for search will have to find ways to improve accuracy and currency of the underlying models and provide caveats to users about potential bias and inaccuracy. Meanwhile, Google, which itself has significant AI systems and probably the best, biggest data sources to train them on, can easily add a conversational interface.

To date, ChatGPT has been offered for free use (to gain experience, publicise capabilities and refine the models), but this is likely to change very soon. OpenAI does not yet have in place an advertising supported model like Google and is likely to first try subscriptions. But when it is no longer free, other competitors will spring up.

One smaller but interesting player is looking to offer the best of both worlds, starting now. This is Andi (andisearch.com). Andi search lets you use GPT style prompts and provides a summary answer (much like ChatGPT), but also provides references and search results on the right to allow validation or further exploration. This is very promising! It should be an exciting time in search this year.

Dealing with Change

We probably all feel a little battered by the levels of change we are experiencing. Technology, pandemic, business models, social mores, ethics, sustainability, legislation and more. It is hard to retain our sense of perspective and balance and self worth when everything seems to be shifting around us!

As architects we are often the agents of change for the organisation, processes, products, systems and technology. But that does not mean we ourselves are always that happy with change! The threat is that it brings risk: Are we focussing on the right things? Are there new factors we aren’t aware of? Is our “known good solution” still relevant?

I find comfort in Jeff Bezos approach which advocates:

“Find what is not going to change and optimise for that”

He recommended, in the case of Amazon, that the following factors were unlikely to change:

  • Customers want cheaper prices

  • Customers want fast delivery

  • Customers want increased selection

And in the case of Amazon Web Services:

  • Customers want reliability

  • Customers want low prices

  • Customers want rapid innovation in adding APIs (increasing utility of the platform)

Find the things in your business / industry that will not change and optimise for them. 

Architecture as a Context for Agility

Agility requires doing focussed things rapidly. The more you know going in, the better decisions you can make quickly. The more you document what you learn, the more knowledge is available for future efforts. Good agile work fills in more of the picture thereby enabling all teams.

The more of the picture is filled in the more we can avoid wasted effort, align our efforts and deliver with less risk. You can’t create the full picture quickly, which is why many agilists avoid architecture.
But you can start with a “paint by numbers” reference model/ontology, which gives you the framework into which to rapidly record your growing knowledge and which indexes where to look for information for your next effort, and what touches the squares you want to colour, so you know how to be informed and compatible.

Every project (agile or otherwise) should:

  • Be informed by our knowledge of current architecture assets and challenges

  • Contribute to an improvement in assets, condition, effectiveness and future readiness

  • Improve the architecture of the portfolio

  • Deliver business value

  • Fill in more of the architecture “big picture” to inform future projects

The environment should:

  • Have a conherent integrative meta model/shared concepts

  • Encourage good work through well conceived principles

  • Have standards for how things get recorded (artefacts) so they are meaningful and sharable

  • Provide a collaborative repository that holds things and makes them findable

#agile #project #architecture #context

If data is the lifeblood, how’s your heart?

Organisations are paying more attention to data management, often driven by compliance, privacy or cyber security concerns. But simply holding data doesn’t generate value.

We need a thorough understanding of the relationships between data (numbers, text, pictures, audio, video, facts), information (data meaningful to humans: salary, sales, order, invoice, fingerprint etc), knowledge (richly connected data: contextual data, trend data, inference) and wisdom (deep insights, experience shaped). Value increases as we move up this hierarchy. Alongside that, if we are to understand what we have, manage it properly, secure it, use it, integrate it etc., we need meta data: data about data. Where is it from? How is it structured? Who owns it? How much can we trust it? How is it derived? What format is it in? Where do we keep it? How long should we hold it? What are the constraints on its use…

All of the above are complicated by the explosion of data brought on by new forms of data; technology capabilities to capture, store, manipulate, communicate, generate, represent and analyse data and innovative applications. Virtualisation of products and services compounds the problem, as more of what we offer and sell to customers is information rather than physical.

There are more opportunities than ever before to profit from data, information, knowledge and its proper use. But there are also more challenges associated with doing it properly, successfully, reliably and securely. All of these rely upon skills and capabilities. Specifically, we need high skills to understand, analyse, model, design and implement data related products and services. This is the realm of Data and Information Architecture.

Architects also need to understand business requirements, facilitate communication and build consensus, define vision, bridge gaps and scope initiatives. They need to guide projects and solution designers. Crucially, they need to connect the business/conceptual view of data with the logical (application) and physical (database and technology) views. They need to devise, apply and encourage use of good principles to evolve the data and information landscape in positive ways.

Data management is ultimately a business responsibility, but can be assisted by many technical skills, including: maturity assessment, modelling, meta data management, technology architecture, risk analysis, integration design and considerations of security and privacy.
A comprehensive data/information architecture and data management capability is vital to deliver business benefits as well as ensure security, privacy and acceptable risk.

These are all topics covered in depth in our Techniques and Deliverables of Information Architecture intensive online live course from the 7-11th November. See details here.

#dataarchitecture #informationarchitecture #digitaltransformation #bigdata #businessintelligence #bi #datamodelling

What comprises a “Solution Architecture”?

A solution is a combination of components in a configuration that solves a problem or exploits an opportunity in a way that meets our goals. Hopefully it is also effective, efficient, sustainable, ethical and relatively risk free.

It is not just a software system, but rather a combination of software, process, people skills, data and technology that meets business, human, technical and legal requirements.

Considering the provided rich picture:

The items outside the circle represent the context in which a solution is developed. We ignore these at our peril. If we do not know the Business Goals for a solution, we can only meet them by extraordinary luck. If we do not know the Legal constraints we will run foul of the law. If we do not understand the Customer and the Stakeholders, we are unlikely to provide something they are happy with. If the service is not delivered via the required Channels or compatible with the Brand strategy, we may miss the mark entirely. In short, many of these should inform our Requirements. Alan Kay famously remarked: Context is worth 80 IQ Points.

Our requirements should also certainly include the Functions that must be performed, the Business Objects (Data) that is used, the Technology we need, the Process to deliver Value, the Application Services we may use or provide, the User Interfaces, the Events we need to respond to or generate and the Locations where we need to be available.

Non-Functional requirements will also play a major role in the viability and success of a solution. These include aspects such as security, reliability, performance, cost, maintainability, flexibility, ease of use, compatibility and many other factors.

Customer Experience is crucial to ensure wide and willing acceptance and delivery of business value. Staff experience is also key, as it that of other professionals who will deal with the solution, including Operations, Support and Maintenance staff.

The Solution Architecture should follow some important principles, including: Modularity, loose coupling, message based communication and open standards. Its also good to have tests built in and automatically repeatable, an affinity for DevOps or Continuous Deployment. User interfaces that are intuitive and built in tutorial aids are really important too.

A cost effective system might be composed of off the shelf components, reusable library elements, configurable components and custom developed elements. The solution includes these and the other elements of human skill and capability, supporting technology, infrastructure, documentation etc.

We may also need to contemplate the development/ implementation dependencies and partition the solution into an initial Minimum Viable Product plus one or more incremental delivery releases to get us to the full capability required.

Solution Architecture is a challenging but very rewarding role.

#SolutionArchitecture #Architecture #Requirements

Just one API?

GraphQL provides single query for queries and updates

Jargon buster at the bottom of post.

First came RPC to call a function across a network. But it was language specific and lacked standard facilities. So DCE was made to address common requirements, such as directory services, time, authentication and remote files. But it was not object oriented when Smalltalk, C++, Java et al arrived. So Microsoft devised DCOM to provide distributed services for Ms languages while others backed CORBA which provided cross platform and cross language services. Both required agreement for message formats ahead of time.

Enter Web Services, leveraging XML to serialise data, WSDL to describe services, UDDI to publish, find and bind to them, and SOAP to message remote objects. Great! We could now find, bind to and invoke services without prior design agreement. But, it was not very efficient and required a lot of plumbing on each end, and quite a bit of knowledge from developers.

So, Roy Fielding devised REST exploiting HTTP to provide a simple way of working with remote Resources. REST allows us to simply access remote servers and retrieve something GET, inform about something POST, store something PUT, update something PATCH or delete something DELETE. This is achieved by creating simple headers and a request line including the URL and parameters. Post also has a body.

REST is very light weight and does not need much infrastructure. Combining it with JSON made it very easy to use from within web pages and mobile applications and it quickly took off.

But there was a problem. Each REST request would get a specific thing from the server. If there is a rich database or knowledge graph on the server, we can create many REST APIs: At least one for each kind of domain object (e.g Customer, Product, Account, Invoice etc. ); Often more than one to cater for different application requirements (partial records, related records etc. ). Plus we will have different APIs to query, to store, to update etc. So, a server with a database managing a score of domain concepts could quickly require 100s of APIs. Ew, that’s a lot of development, testing, deployment, documentation, maintenance…

Facebook ran into this problem at scale. Their solution was a query language that would live in the server as a single entry point and receive a query request as a parameter. This is not dissimilar to the way a relational database receives dynamic SQL requests. Now the tailoring of a response can happen in the server (more efficient) and we have only one API endpoint to maintain. Voila. So that solved the problem for Facebook… Fortunately, they published it as GraphQL which allows writing query and update (mutate) statements and having these fulfilled by a suitable GraphQL processor / application / database on the server. Initially, these were discrete, but they are starting to be embedded in database systems, especially Graph Databases. One good example is DGraph.

JARGON BUSTER:

You can also find good explanations of most of these topics on Wikipedia

  • RPC - Remote Procedure Calls

  • DCE - Distributed Computing Environment

  • API - Application Programming Interface. A way of requesting a service or function contained in another piece of software. Most commonly used today to refer to a REST API

  • COM+ - Microsoft Component Object Model. An architecture that allowed sharing of objects between Microsoft languages.

  • DCOM - Microsoft distributed COM. Similar to COM+, but allowing objects to be remote

  • .Net - Microsoft Component model and framework that succeeded COM and DCOM

  • CORBA - Common Object Request Broker Architecture. An architecture for distributed object messaging across languages and technologies.

  • Web Services - A set of standards, leveraging XML, that allows requesting services across the Internet. Includes WSDL, UDDI, SOAP.

  • XML - eXtensible Markup Language. A standard for encoding data onto text with specific tagging of the meaning of the values.

  • WSDL - Web Services Description Language. An XML document describing a Web Service.

  • SOAP - Simple Object Access Protocol. A way to invoke a (remote) service in the Web Services approach. Effectively an XML message requesting a given service and expecting an XML response message.

  • UDDI - Universal Description Discovery and Integration. A protocol for publishing Web Service Descriptions and for finding these.

  • HTTP - Hypertext Transport Protocol. The protocol of the internet which allows hyper-linking.

  • REST - Resource State Transfer protocol. A protocol that leverages the HTTP intrinsic functions to support requesting services across the Internet with minimal other infrastructure.

  • JSON - Javascript Object Notation. A way of encoding JavaScript data structures on to text for transmission or sharing. Similar purpose to XML, but lighter weight.

  • GraphQL - A query language used on a client and interpreted in a server which allows easy retrieval of data using graph concepts (Nodes, properties and relationships).

  • RDF - Resource Description Framework. A standard for defining facts and knowledge using simple statements with a Subject, Predicate, Object format. Part of Semantic Web standards.

  • DGraph - a Property Graph database that supports graph schemas, RDF, JSON and GraphQL natively at web scale. Also does ACID transactions.

  • ACID - Atomic, Consistent, Isolated, Durable. Desirable attributes of transactions in a database.

#API #Services #REST #WebServices #SolutionArchitecture #Design #GraphQL