Tag Archives: institution

In IoT ecosystem evolution, constraints = opportunities for IoT innovators

Leave a reply

What are our opportunities for guiding the rapidly evolving IoT ecosystem? The Internet of Things, with its explosive growth, unprecedented variety of device & system types, lack of broadly shared language and conceptual frameworks to discuss and plan, lack of precedence for implementation, and the organizationally complex consumer systems — i.e. cities and institutions — required to implement and manage these IoT systems — all make for a challenging space. It can be difficult to even know where to start. One way to add structure and framework to the conversation is to introduce some constraints — and good news! There are constraints already there! They’re just not broadly seen or talked about yet.

What does a successful IoT system implementation look like ?

A natural source for constraints is from those things that define a successful IoT System implementation in an institution or city. I use two primary components to define IoT System implementation success:

  1. Return on Investment (ROI)
  2. Cyber risk profile

Regarding the first — ROI, does the system do what we thought it would do at the costs/investment that we thought would be incurred? As discussed in a recent post on IoT System costing, determining costs of IoT Systems implementation is different from traditional enterprise systems. Most institutions and cities have little experience at it and are generally not very good at it. Further, other subtleties such as expectations of the data created from deployed IoT systems across a spectrum of populations, demographics, & constituencies directly impact perceptions of system (and investment) success.

Regarding the second — cyber risk profile, did the IoT System implementation make things worse for the institution or city? Cyber risk profile degradation can come from poorly configured devices/endpoints, insufficient management resources (skill, capacity) for endpoints and data aggregators/controllers, inadequate vendor management, and others.

Constraints drive opportunities in the IoT ecosystem

These same two analysis requirements of a city’s or institution’s success, aka constraints, can also be used by innovators and providers of IoT systems. Knowledge of these constraints by IoT systems providers, these requirements for city/institution implementation success, creates opportunity for the IoT systems innovator and provider by identifying where they can help address organizational complexities in the course of pursuing ROI and cyber risk posture/profile objectives.

IoT systems are different

As discussed in other articles and posts, IoT Systems are different. The process of selecting, procuring, implementing, & managing IoT systems is different from doing the same for traditional enterprise systems such as email, calendaring, resource and customer management, etc. At least six aspects of IoT Systems contribute to this difference:

  1. High number and growth rate of IoT devices
  2. High degree of variability of device types & variability of multiple hardware/software components within a device
  3. Lack of language and frameworks to discuss IoT opportunities, risks
  4. IoT Systems span multiple organizations within an institution or city
  5. IoT endpoint/devices tend to be out of sight out of mind
  6. Lack of precedent for successfully implementing these systems, few examples, few patterns to follow

Of these differences, #4 – the organizational spanning aspect of IoT Systems — presents a subtle but substantial challenge. Deploying IoT Systems in a city or institution is not like deploying an enterprise application in a data center or SaaS in the cloud and then providing for end-user training and support. This, of course, does not mean that deploying large enterprise systems is easy by any stretch, but rather that there are more and different organizations required in the technical, operational, and management aspects of the system.  Because of this, new levels of inter-organizational cooperation and collaboration are sought. And, as we all have experienced, collaboration and cooperation is frequently touted but successful collaboration and cooperation is often not achieved — “the discrepancy between the promise of collaboration and the reality of persistent failure” (Koschmann).

Cities and institutions are complex multi-component organizations that offer a complex substrate for IoT System implementation. These complex IoT product and service consuming organizations are not blank slate, clean whiteboard, or powerpoint deck solution organizations. There is little homogeneity here.

IoT Systems innovators and providers that recognize these constraints brought on by these complex consumer systems, that seek to learn the institutional organizational challenges in detail, and get in the dirt at the outset with the city or institution will ultimately be IoT Systems ecosystem drivers.

“I built it in my garage, it works there, it’ll be awesome in your city!”

Because of the seemingly unbounded potential of IoT Systems solutions, there’s also room for undifferentiated, poorly provisioned, and poorly serviced garbage in this space.

Because of the newness of IoT Systems, often there are many technologies and many vendors without particularly long track records. There are some big names in the game of course — Cisco, Microsoft, Intel, Siemens for example. But there are many providers in that long tail, both proven and unproven, and some of them will offer great innovation and value. Some of them will not. The challenge for institutions and cities is to work to separate the wheat from the chaff as they select, procure, implement, and manage IoT Systems.

Going by name brand alone is not sufficient because there will be many new innovators and providers that do indeed offer promising and solid solutions that give a reasonable likelihood of ROI and an approach that does not degrade the existing cyber risk profile of the institution. Further, sometimes large companies can be problematic because they are used to throwing their weight around, possibly invested heavily in particular approaches, and may not be open to new or alternative approaches. This may or may not be with whom a city or institution wants to work.

Eyes off the bling for a moment

So how can a city or institution begin to separate the wheat from the chaff in choosing IoT systems? An initial step can be to take one’s eyes off of the ‘bling’ for a moment. The bling is all of the feature sets and bells and whistles that most think of when they think of IoT systems. So, a three step process would be:

  1. Take eyes off of the bling (feature sets, bells & whistles) for a moment
  2. Review implementation challenges internal to the institution
    • organizational spanning complexity
    • calculating IoT system support costs across all organizations
    • analyzing internally available skill sets and capacity
    • consider what criteria different demographics will use to assess success or failure
    • seek input in estimating cyber risk to which an  institution or city is already exposed to provide an estimated baseline
  3. Seek and prioritize IoT Systems innovators/providers that help address some of these internal organizational challenges and shortcomings

insideoutoutsidein

Cities and institutions look inside out — Some of their internal challenges include:

  • organizational complexity (spanning)
  • IoT system support staffing capacity
  • appropriate skill sets
  • IoT system support tools availability & experience
  • what ROI will a particular provider’s IoT system bring?
  • will implementing this IoT system make my cyber risk picture worse? how do I know?

IoT innovators & providers can look outside in —  and use these constraints to create market differentiators for their organizations, such as:

  • can I help city/institution address internal challenges?
  • can I provide tools to help them manage their system?
  • can I help them reach the ROI they were expecting?
  • can I help them mitigate their cyber risk from this implementation?

Not just one IoT System

We’ve been talking about just one prototypical IoT System for an institution or city. In practice, institutions and cities will have many IoT Systems. Many of these  IoT Systems will:

  • use shared technical resources of the city or institution, eg network and supporting systems
  • have interdependency with other systems
    • at device level
    • at data level
    • to include co-existing with legacy systems & new systems
  • dip into the same limited pool of skill sets and capacity for systems support

This further deepens the IoT Systems management challenge within the city or institution. Implementation challenges for these complex city and institutional consumers will only continue to grow. They won’t diminish.

IoT Systems innovators and providers that recognize and speak to this additional level of complexity — this ecosystem with multiple providers and vendors within an institution —  and provide options, services, and support to help cities and institutions manage this complexity will set themselves apart from the competition and develop longer lasting relationships.

In this seemingly open-ended space of IoT systems possibilities, identifying and developing solutions for organic complex consumer constraints and challenges can be a differentiator for IoT product innovators and service providers.

Costing IoT Systems in cities & institutions

Leave a reply

Determining the total cost of ownership, operation, and stewardship of IoT Systems for an institution or city has a number of considerations. Some of these considerations are shared with traditional enterprise systems and some are unique to IoT Systems. Lack of realization and acknowledgement of these costs can lead to disappointing and costly IoT Systems outcomes. These disappointing outcomes manifest themselves in the lost ROI of an IoT investment as well as negative impacts to the cyber-risk profile to an institution.

Listing from least complex/nuanced to more complex/nuanced …

  1. Costs to host servers, databases, & redundancy — whether under desks (hopefully not), in local data centers, or in the cloud
  2. Costs to support large numbers of geographically distributed ‘Things’ & devices (the T in IoT) & the different institutional organizations that may be involved (some of which may not see or realize the potential benefit to the institution but may bear at least some of its support costs)
  3. Costs stemming from the natural friction, sometimes small & sometimes large, between multiple historically disparate organizations within an institution as they attempt to coordinate, collaborate, and address problems of understanding to support the system

1. The more familiar part — application server, database server, & redundancy costing

This costing is more closely aligned with traditional enterprise application costing than other IoT systems costs. Application licensing and support agreements are a part of this aspect. Important additional costs to include, though, are: what are the costs of hosting those applications and databases? And what are the costs of supporting that hosting (e.g. who manages the relationships, the tickets, the problems, etc).

The applications servers or virtual machines (VM’s), database servers/VM’s, and redundancy servers/VM’s can be hosted in a local data center, shared data center, the cloud, or similar. Hosting or otherwise servicing these applications and supporting databases have their own costs. In addition to fees for the hosting, there’s also some cost to managing the vendor relationship and agreement/contract.

2. A guy, a truck, and a ladder – a less obvious part of IoT System costing

 

this is not cheap

this is not cheap

Imagine an institution or city that implements 500 smart street lights that provide lighting, sense movement, maybe samples air quality, and possibly monitors and reports street or underground vibration. There will be some failure rate amongst the components in any single device/endpoint and some failure rate across the total number of installed devices/sensors – the T’s in IoT.

In this hypothetical scenario, troubleshooting and/or repair means:

  • deploying 1 or more skilled tradespeople
    • 1 or more for the work & possibly another as a safety observer
  • rolling a truck or trucks
    • with associated vehicle/fleet/fuel costs
  • spending 1-2 hours just to get to the point of troubleshooting/repair
  • 2 – 4 hours troubleshooting/repair
  • 1-2 hours wrap up and return

For this hypothetical example, let’s say the skilled tradespeople involved make $60/hour and their benefit load (expense to institution or city) is 25% for a total $75/hour expense. So, disregarding fleet and related costs, one estimate might be:

(2 hrs prep + 4 hrs on site troubleshooting/repair + 2 hrs recovery) x 2 tradespeople x $75/hr = $1200 per support event

Continuing on the thread, let’s say there’s a 10% / year failure rate (or required hardware/software update rate) of at least some component on a single device or Thing (T in IoT). That would be:

500 devices x 10% repair or troubleshoot rate/year x $1200/event = $60,000 / year

That dollar figure starts to become non-trivial. And that’s just one IoT System. Cities and institutions will have many, a portfolio, of well-managed or less-well-managed IoT Systems. Another hypothetical example is in the example  below —

IoTSysCostingExample050917

multiple costs involved

3. The insidious part — loss from organizational friction

The least apparent and possibly most costly aspect is the loss that occurs from the coordination and collaboration required between organizations and the oversight needed across all of them for a successful implementation. In the idealized scenario of institutional capacity, there is a homogeneous set of resources that include components such as available staffing levels (FTE), requisite skill sets (technical, operational, and interpersonal), support funding, political/institutional will to support the implementation and operation of the IoT System, vendor relationships, and other.

idealizedcapacity

idealized view of IoT System management capacity

In practice, however, this institutional capacity is comprised of many different organizations and the interrelationships between them.  While collaboration and inter-organizational cooperation is typically universally lauded, we all know from personal and professional experience that often collaboration between institutional organizations does not in fact work so well. Research has also been done on this phenomena where, “the discrepancy between the promise of collaboration and the reality of persistent failure” is studied (Koschmann).

Wherever two or more organizations interact with each other, some sort of friction or system loss is present. Metaphorically similar to the 2nd law of thermodynamics, not all of the time, energy, resources put into ostensible institutional IoT System management capacity will be used, or can be used, to do the expected work. In the organizational friction example, losses can come from a multitude of sources where the number of friction sources and the intensity of each can vary from organizational relationship to organizational relationship. Examples of this sort of friction/system loss and resultant loss in expected institutional capacity include:

real capacity stems from multiple organizations with multiple relationships and the friction between

real capacity stems from multiple organizations with multiple relationships and the friction between

The insidious part is that while the friction between any two organizations, may be small and possibly not obvious, these small instances of organizational friction aggregate across the whole institutional and IoT System implementation effort. Further, not all relationships are one-to-one. There are often many relationships where many organizations are involved. Likening to Newton’s Three Body Problem, adding a third planet, billiard ball, or organization can significantly increase the complexity of analysis, prediction/forecasting, and the ability to get work done.

capacityloss

 

Costing differently

In practice, capturing all of these costs can be challenging. #1 — application and database licensing and hosting costs is relatively the most straightforward and for which we generally have the most experience, but that doesn’t mean it’s effortless. We have less experience with #2 for IoT Systems — the guy, truck, & ladder costs — but that support cost can be estimated and the failure rates of devices and device components can be estimated. Costing #3 — aggregated inter-organizational friction is, by far, the most difficult and possibly the most impactful — in part because of its magnitude and in part because of the uncertainty it introduces.

The import thing, I believe, is to acknowledge all of these components, compute and estimate what we can, and work to allow for and hopefully prepare for the unique uncertainty that selecting, procuring, implementing, and managing IoT Systems brings. If we do this work, we have our best chance of reaching anticipated ROI and not degrading (possibly even enhancing) the risk profiles of our cities and institutions.

 

 

Institutional considerations for managing risk around IoT

Leave a reply
socket

sockets for vendor products & services

There are a number of things to think about when planning and deploying an IoT system in your institution. In posts here since last spring, several issues have been touched upon — the idea of sockets and seams in vendor relationships, the rapid growth in vendor relationships to be managed and the resulting costs to your organization, communicating IoT risk, some quick risk visualization techniques based on Shodan data, initial categorization of IoT systems, and others.  The FBI warning on IoT last week is a further reminder of what we’re up against.

There is a lot to chew on and digest in this rapidly changing IoT ecosystem. Below is a partial list of some things to consider when planning and deploying IoT systems and devices in your institution. It’s not a checklist where all work is done when the checking is complete. Rather, it is intended to be a starting list of potential talking points that you can have with your team and your potential IoT vendors.

Some IoT Planning Considerations

  • Does IoT vendor need 1 (or more) data feeds/data sharing from your organization?
    • Are the data feeds well-defined?
    • Do they exist already?
    • If not, who will create & support them?
    • Are there privacy considerations?
  • How many endpoint devices will be installed?
    • Is there a patch plan?
    • Do you do the patching?
    • Who manages the plan, you or the vendor?
  • Does this vendor’s system have dependencies on other systems?
  • How many IoT systems are you already managing?
    • How many endpoints do you already have?
    • Are you anticipating/planning or planning more in the next 18 months?
  • Is there a commissioning plan? Or have IoT vendor deliverable expectations otherwise been stated (contract, memorandum of understanding, letter, other?)
    • Has the vendor changed default logins and passwords? Has the password schema been shared with you?
    • Are non-required ports closed on all your deployed IoT endpoints?
    • Has the vendor port scanned (or similar) all deployed IoT endpoints after installation?
    • Is there a plan (for you or vendor) to periodically spot check configuration of endpoint devices?
  • Has the installed system been documented?
    • Is there (at least) a simple architecture diagram?
      • Server configuration documented?
      • Endpoint IP addresses & ports indicated?
  • Who pays for the vendor’s system requirements (eg hardware, supporting software, networking, etc?)
    • Does local support (staffing/FTE) exist to support the installation? Is it available? Will it remain available?
    • If supporting IoT servers are hosted in a data center, who pays those costs?
      • startup & ongoing costs?
    • Same for cloud — if hosted in cloud, who pays those costs?
      • startup & ongoing costs?
  • What is total operational cost after installation?
    • licensing costs
    • support contract costs
    • hosting requirements costs
    • business resiliency requirements costs
      • eg redundancy, recovery, etc for OS, databases, apps
  • How can the vendor demonstrate contract performance?
    • Okay to ask vendor to help you figure this out
  • Who in your organization will manage the vendor contract for vendor performance?
    • Without person/team to do this, the contract won’t get managed
  • Can vendor maintenance contract offset local IT support shortages?
    • If not, then this might not be the deal you want
  • For remote support, how does vendor safeguard login & account information?
    • Do they have a company policy or Standard Operating Procedure that they can share with you?
  • Is a risk sharing agreement in place between you and the vendor?
    • Who is liable for what?

Typically, with the resources at hand, it will be difficult to get through all of these — maybe even some of these. The important thing, though, is to get through what we can and then be aware of and acknowledge the ones we weren’t able to do. It’s way better to know we’ve come up short given limited resources than to think we’ve covered everything when we’re not even in the ballpark.