Tag Archives: complexity

Bathtubs, manageability, & IoT

The limited funding and staffing resources inherent in almost all institutions and cities creates a delicate balance between IT systems operations, managing institutional risk, and cybersecurity operations. A critical component to this balance is systems manageability. Implementing unmanaged/under-managed systems can quickly perturb this balance and cause reactionary spending, such as on cybersecurity incident response, institutional reputation damage control, unplanned systems repair dollars, as well as others.

IoT Systems — with their multi-organizational boundary spanning, unclear systems ownership and accountability, lack of precedence for implementation, and high number of networked computing devices (‘Things’) — are particular candidates for unmanaged/under-managed systems in a city or institution. 

Systems manageability

IT systems that tend to be more manageable allow for more predictability in an institution’s resource and cashflow planning.  Criteria for high systems manageability include:

  • having well-defined performance expectations
  • thoughtful and thorough implementation
  • accessible training and documentation
  • strong vendor support
  • others

Unmanaged or under-managed systems increase the likelihood of a cyber event such as device compromise or whole system compromise as well as facilitate potentially substantial operations disruption and unplanned financial burden. 

Bathtub modeling

We can use some concepts from stocks and flows diagrams where the stock is represented by a bathtub to create a basic model of resource availability in this delicate dance of balancing of resources for IT systems operations, cybersecurity operations, and managing institutional risk.

My understanding that the use of a bathtub to represent stocks and flows goes back to 2000 when John Sterman and Linda Booth Sweeney published results of an experiment on how people understand and interpret complex systems. On a related note, I found the book, Thinking in Systems, by the late Donella Meadows to be a very consumable and helpful introduction to stocks and flows diagrams.

bathtub metaphor for stocks and flows

The idea is that the ‘stock’ is the level of water in the tub. Water can flow into the tub, raising the tub level, and that amount can be varied by some mechanism(s) or external constraints. Similarly, water can flow out of the bathtub, draining the tub, and there is a mechanism for controlling the rate of that outflow. And, of course, both could happen at the same time.

Bathtub of bucks

inflows of $$ increase the tub level, outflows of $$ decrease the tub level

Now, imagine that instead of water, the tub holds metaphorical dollars. The tub can be thought of as an account, a set of funds, ‘budget number’, set of budget numbers, or similar. The inflows then are one or more sources for adding dollars to that tub with a mechanism or set of constraints that determines the rate of flow into the tub. Similarly, there is a mechanism for setting how much flows out of the tub (spending or investing).

City and institutional spending

Cities and institutions have multiple sources of inflows, most of which they probably don’t control. Those inflows have independent characteristics from each other as well as some interdependencies with each other. The main takeaway is that the city or institution probably does not control a whole lot regarding what’s coming in.

The spending from the top tub can go to multiple places, themselves other tubs. From the top bathtub, most organizations make decisions between operational dollars (running things) and capital dollars (buying or building big things).

splitting between operational $$ (running things) & capital $$ (buying or building things)

IT & cybersec resources & spending

From the operational dollars tub, some funding goes to IT operations, some goes to cybersecurity operations (eg CISO’s office), and other funding goes to many other traditional and important areas such as HR, finance, policy/law enforcement, and others.

Operational dollars, in turn, gets disbursed across multiple other operational tubs

In the interest of keeping the diagram simpler for our discussion, we won’t include capital spending or non-IT/cybersec spending in subsequent diagrams.

IT systems services and cybersecurity services

Funds from the IT operational bathtub are used to resource the management of various IT systems and sub-systems in the institution or city. This includes both on-premise systems as well as cloud-based systems. Examples include enterprise resource management (ERP) systems, institutional learning/training systems, calendaring and email systems, and others.

Systems that have known performance expectations and implementation precedents (either themselves or peer implementations) can provide the basis for a fairly reasonable calculation to be made on required staffing and funding support requirements.

Similarly, the city/institutional department/organization providing information security services  (usually the CISO’s office) also has a set of well-managed services that are planned for and delivered. Examples of these information security services might include: education and outreach, incident management capability, privacy policy guidance, intelligence analysis, and others. The CISO’s office will work to develop services and capabilities based on the IT systems that the city or institution is operating, known and evolving threats and vulnerabilities, existing risk levels, and others.

resourcing planned and reasonably well-managed systems and services

 

The trouble with unplanned, under-managed, and unmanaged systems

Managing and identifying management support resources can be challenging enough with known systems. Challenges and institutional risk quickly becomes exacerbated though when unplanned or weakly planned systems are added. For example, after the budget/planning cycle, an influential person or group may decide that the city or institution “must have” System X. And then later someone else with influence might insist on (unplanned) System Y.

When these unplanned or under-planned systems are added, several deleterious things can happen:

  • the unplanned system drains from the IT operational funding tub in the forms of implementation staffing, management staffing, and support tools  
  • planned systems now no longer have their expected resources and they themselves can become under-managed in addition to the add-on system that is very likely also to be under-managed
  • institutional/city risk increases because unmanaged/under-managed systems increase likelihood of system comprise due to misconfiguration, mismanagement, lack of oversight, failure of (or lack of application of) controls
  • things get worse as the problem also transmits to a different bathtub, ie the information security services provider for the city or institution, eg the CISO’s office
  • when compromise occurs — particularly on systems that the CISO’s office could not plan for — the CISO’s office is now forced to work in a reactionary mode. This is expensive and pulls resources from planned cybersecurity services

unplanned, under-managed systems also transmit their problems to the CISO’s office in the form of increased likelihood of systems compromise

IoT Systems often fall into the unplanned, under-managed category

Several aspects of IoT Systems deployments can contribute to them having high risk of being weakly planned and under-managed systems —

  • lack of precedent for implementation & management
    • cities/institutions don’t have deep experience with these systems
      • true for all phases – systems selection, procurement, implementation, & management
    • few, if any, peer cities/institutions from which to learn for systems expected to last years or decades (sufficient time hasn’t gone by)
  • accountability and ownership unclear
    • IoT systems span many organizations within a city or institution
    • most organizations are not familiar or practiced at coordinating with each other in this role
  • acquisition path – IoT Systems can come into the institution through many non-traditional paths
    • these IoT Systems are rarely acquired by central IT
    • even if acquired through central IT, traditional systems vetting approaches not sufficient
  • no established vetting of IoT systems prior to purchase
    • performance expectations unknown or unclear (see ownership above)
  • the city or institutional department acquiring the system might not be the one supporting the system
  • Newness and rapid evolution IoT Systems makes them hard to discuss, categorize, and plan for

the newness, novelty, and rapid evolution of IoT Systems will continue to make very susceptible to being under-managed systems

Rapid evolution of IoT Systems vs glacial pace of institutional change

While there are no silver bullets or magic technologies (and we shouldn’t spend much time looking for them) to address these added risks that IoT Systems bring, there are things that we can do now, or at least begin now, that can positively impact our risk exposure as institutions and cities. While we’re interested in mitigating risks that we have now from IoT Systems, the impact of IoT systems in our cities and institutions in the future will be much higher.

Some things that can be done now include —

  • establish a set of criteria for your city’s or institution’s for IoT Systems
    • circulate and share this across the organization
    • one starting point is here
    • a related document from the Internet2 IoT task force on IoT risk  is here
  • identify IoT Systems ownership and accountability
    • require before acquisition
  • identify institutional language used to communicate traditional risk & incorporate that into IoT risk conversations, guidelines, and planning
  • consider an IoT Systems oversight group for your city or institution

Making broad changes, perception changes, and policy changes in cities and institutions is arduous work that takes time, leadership, political capital, and patience.  It is important that we begin now because this level of institutional change will likely take some time and the impact of not making the changes is increasing rapidly.

 

In IoT ecosystem evolution, constraints = opportunities for IoT innovators

What are our opportunities for guiding the rapidly evolving IoT ecosystem? The Internet of Things, with its explosive growth, unprecedented variety of device & system types, lack of broadly shared language and conceptual frameworks to discuss and plan, lack of precedence for implementation, and the organizationally complex consumer systems — i.e. cities and institutions — required to implement and manage these IoT systems — all make for a challenging space. It can be difficult to even know where to start. One way to add structure and framework to the conversation is to introduce some constraints — and good news! There are constraints already there! They’re just not broadly seen or talked about yet.

What does a successful IoT system implementation look like ?

A natural source for constraints is from those things that define a successful IoT System implementation in an institution or city. I use two primary components to define IoT System implementation success:

  1. Return on Investment (ROI)
  2. Cyber risk profile

Regarding the first — ROI, does the system do what we thought it would do at the costs/investment that we thought would be incurred? As discussed in a recent post on IoT System costing, determining costs of IoT Systems implementation is different from traditional enterprise systems. Most institutions and cities have little experience at it and are generally not very good at it. Further, other subtleties such as expectations of the data created from deployed IoT systems across a spectrum of populations, demographics, & constituencies directly impact perceptions of system (and investment) success.

Regarding the second — cyber risk profile, did the IoT System implementation make things worse for the institution or city? Cyber risk profile degradation can come from poorly configured devices/endpoints, insufficient management resources (skill, capacity) for endpoints and data aggregators/controllers, inadequate vendor management, and others.

Constraints drive opportunities in the IoT ecosystem

These same two analysis requirements of a city’s or institution’s success, aka constraints, can also be used by innovators and providers of IoT systems. Knowledge of these constraints by IoT systems providers, these requirements for city/institution implementation success, creates opportunity for the IoT systems innovator and provider by identifying where they can help address organizational complexities in the course of pursuing ROI and cyber risk posture/profile objectives.

IoT systems are different

As discussed in other articles and posts, IoT Systems are different. The process of selecting, procuring, implementing, & managing IoT systems is different from doing the same for traditional enterprise systems such as email, calendaring, resource and customer management, etc. At least six aspects of IoT Systems contribute to this difference:

  1. High number and growth rate of IoT devices
  2. High degree of variability of device types & variability of multiple hardware/software components within a device
  3. Lack of language and frameworks to discuss IoT opportunities, risks
  4. IoT Systems span multiple organizations within an institution or city
  5. IoT endpoint/devices tend to be out of sight out of mind
  6. Lack of precedent for successfully implementing these systems, few examples, few patterns to follow

Of these differences, #4 – the organizational spanning aspect of IoT Systems — presents a subtle but substantial challenge. Deploying IoT Systems in a city or institution is not like deploying an enterprise application in a data center or SaaS in the cloud and then providing for end-user training and support. This, of course, does not mean that deploying large enterprise systems is easy by any stretch, but rather that there are more and different organizations required in the technical, operational, and management aspects of the system.  Because of this, new levels of inter-organizational cooperation and collaboration are sought. And, as we all have experienced, collaboration and cooperation is frequently touted but successful collaboration and cooperation is often not achieved — “the discrepancy between the promise of collaboration and the reality of persistent failure” (Koschmann).

Cities and institutions are complex multi-component organizations that offer a complex substrate for IoT System implementation. These complex IoT product and service consuming organizations are not blank slate, clean whiteboard, or powerpoint deck solution organizations. There is little homogeneity here.

IoT Systems innovators and providers that recognize these constraints brought on by these complex consumer systems, that seek to learn the institutional organizational challenges in detail, and get in the dirt at the outset with the city or institution will ultimately be IoT Systems ecosystem drivers.

“I built it in my garage, it works there, it’ll be awesome in your city!”

Because of the seemingly unbounded potential of IoT Systems solutions, there’s also room for undifferentiated, poorly provisioned, and poorly serviced garbage in this space.

Because of the newness of IoT Systems, often there are many technologies and many vendors without particularly long track records. There are some big names in the game of course — Cisco, Microsoft, Intel, Siemens for example. But there are many providers in that long tail, both proven and unproven, and some of them will offer great innovation and value. Some of them will not. The challenge for institutions and cities is to work to separate the wheat from the chaff as they select, procure, implement, and manage IoT Systems.

Going by name brand alone is not sufficient because there will be many new innovators and providers that do indeed offer promising and solid solutions that give a reasonable likelihood of ROI and an approach that does not degrade the existing cyber risk profile of the institution. Further, sometimes large companies can be problematic because they are used to throwing their weight around, possibly invested heavily in particular approaches, and may not be open to new or alternative approaches. This may or may not be with whom a city or institution wants to work.

Eyes off the bling for a moment

So how can a city or institution begin to separate the wheat from the chaff in choosing IoT systems? An initial step can be to take one’s eyes off of the ‘bling’ for a moment. The bling is all of the feature sets and bells and whistles that most think of when they think of IoT systems. So, a three step process would be:

  1. Take eyes off of the bling (feature sets, bells & whistles) for a moment
  2. Review implementation challenges internal to the institution
    • organizational spanning complexity
    • calculating IoT system support costs across all organizations
    • analyzing internally available skill sets and capacity
    • consider what criteria different demographics will use to assess success or failure
    • seek input in estimating cyber risk to which an  institution or city is already exposed to provide an estimated baseline
  3. Seek and prioritize IoT Systems innovators/providers that help address some of these internal organizational challenges and shortcomings

insideoutoutsidein

Cities and institutions look inside out — Some of their internal challenges include:

  • organizational complexity (spanning)
  • IoT system support staffing capacity
  • appropriate skill sets
  • IoT system support tools availability & experience
  • what ROI will a particular provider’s IoT system bring?
  • will implementing this IoT system make my cyber risk picture worse? how do I know?

IoT innovators & providers can look outside in —  and use these constraints to create market differentiators for their organizations, such as:

  • can I help city/institution address internal challenges?
  • can I provide tools to help them manage their system?
  • can I help them reach the ROI they were expecting?
  • can I help them mitigate their cyber risk from this implementation?

Not just one IoT System

We’ve been talking about just one prototypical IoT System for an institution or city. In practice, institutions and cities will have many IoT Systems. Many of these  IoT Systems will:

  • use shared technical resources of the city or institution, eg network and supporting systems
  • have interdependency with other systems
    • at device level
    • at data level
    • to include co-existing with legacy systems & new systems
  • dip into the same limited pool of skill sets and capacity for systems support

This further deepens the IoT Systems management challenge within the city or institution. Implementation challenges for these complex city and institutional consumers will only continue to grow. They won’t diminish.

IoT Systems innovators and providers that recognize and speak to this additional level of complexity — this ecosystem with multiple providers and vendors within an institution —  and provide options, services, and support to help cities and institutions manage this complexity will set themselves apart from the competition and develop longer lasting relationships.

In this seemingly open-ended space of IoT systems possibilities, identifying and developing solutions for organic complex consumer constraints and challenges can be a differentiator for IoT product innovators and service providers.

What Floyd the Barber knew about information risk management

The Mayberry Model

Watch the till and lock the door at night. If you were opening a small business 30 years ago, your major security concerns were probably to keep an eye on the till (cash register) during the day and to lock the door at night.  It reminds me a little bit of the Andy Griffith Show which ran in the 1960’s about a small fictional town called Mayberry RFD in North Carolina.  Mayberry enterprises included Floyd’s Barbershop, Emmett’s Fix-It Shop, and Foley’s Grocery.

mayberry

Floyd didn’t need a risk management program, much less an IT risk management program to run his business.  It was pretty easy to remember — watch the till and lock the door.   He could also easily describe and assign those tasks to someone else if he wasn’t available.    Further, it was fairly easy to watch the till:  Money was physical — paper or metal — and it was transferred to or from the cash drawer. He knew everyone that came into his shop.  Same for Emmett and his Fix-It Shop.  Plus they had the added bonus of a pleasant bell ring whenever the cash drawer opened.  This leads us to the MISRMP (Mayberry Information Security & Risk Management Plan).

cash_register1

Mayberry Information Security and Risk Management Plan:

  • Watch the till 
  • Lock the door at night
  • Make sure the cash register bell is working

Today’s model

Fast forward to a small business today, however, and we have a different story.  Today, in our online stores selling products, services, or information, there is no physical till and probably little to no physical money.  There are online banks, credit cards, and PayPal accounts and we really don’t know where our money is.  We just hope we can get it when we need it.

There are not actual hands in the till nor warm bodies standing near the till when the cash drawer is opened. There is no soft bell ring to let us know the cash drawer just opened.  We don’t know the people in the store and they don’t go away when the front door is locked.  Our customers shop 24/7.

Further, instead of a till with a cash drawer, our businesses rely on very complex and interconnected equipment and systems — workstations, servers, routers, and cloud services — and we don’t have the time to stop and understand how all of this works because we’re busy running a business.  Floyd’s only piece of financial equipment was the cash register (and Emmett could fix that if it broke).

This new way of doing business has happened pretty fast. It is not possible to manage and control all the pieces that make up our financial transactions.  We also have a lot more financial transactions.  While the Internet has brought many more customers to our door, it has also brought many more criminals to our door.  Making the situation even more challenging, we largely don’t have the tools in place to manage our information risks.

Floyd the Barber

Floyd the Barber

What Floyd knew (and we don’t): 

  • who his customers were (knew them by face and name)
  • what their intentions were (wanted to purchased a haircut or shave or steal from the till)
  • where his money was (in the till, in the bank, or in his pocket while being transferred from the shop to the bank)
  • when business transactions occurred ( 9:00 – 5:00 but closed for lunch and closed on Sundays)
  • what was happening in his store after hours (nothing)

That is to say, Floyd had much less business uncertainty than we must contend with today.  He could handle most of his uncertainty by watching the till and locking the door at night. Our small and medium sized businesses today, though, are much more complex, have much higher levels of uncertainty, and need be risk managed to allow us to operate and grow.

As Floyd managed his security and risk to operate a successful business, so must we — ours is just more complicated.

What are the 3 biggest IT & Information Management risks that you see affecting your business?