Tag Archives: government

Bathtubs, manageability, & IoT

The limited funding and staffing resources inherent in almost all institutions and cities creates a delicate balance between IT systems operations, managing institutional risk, and cybersecurity operations. A critical component to this balance is systems manageability. Implementing unmanaged/under-managed systems can quickly perturb this balance and cause reactionary spending, such as on cybersecurity incident response, institutional reputation damage control, unplanned systems repair dollars, as well as others.

IoT Systems — with their multi-organizational boundary spanning, unclear systems ownership and accountability, lack of precedence for implementation, and high number of networked computing devices (‘Things’) — are particular candidates for unmanaged/under-managed systems in a city or institution. 

Systems manageability

IT systems that tend to be more manageable allow for more predictability in an institution’s resource and cashflow planning.  Criteria for high systems manageability include:

  • having well-defined performance expectations
  • thoughtful and thorough implementation
  • accessible training and documentation
  • strong vendor support
  • others

Unmanaged or under-managed systems increase the likelihood of a cyber event such as device compromise or whole system compromise as well as facilitate potentially substantial operations disruption and unplanned financial burden. 

Bathtub modeling

We can use some concepts from stocks and flows diagrams where the stock is represented by a bathtub to create a basic model of resource availability in this delicate dance of balancing of resources for IT systems operations, cybersecurity operations, and managing institutional risk.

My understanding that the use of a bathtub to represent stocks and flows goes back to 2000 when John Sterman and Linda Booth Sweeney published results of an experiment on how people understand and interpret complex systems. On a related note, I found the book, Thinking in Systems, by the late Donella Meadows to be a very consumable and helpful introduction to stocks and flows diagrams.

bathtub metaphor for stocks and flows

The idea is that the ‘stock’ is the level of water in the tub. Water can flow into the tub, raising the tub level, and that amount can be varied by some mechanism(s) or external constraints. Similarly, water can flow out of the bathtub, draining the tub, and there is a mechanism for controlling the rate of that outflow. And, of course, both could happen at the same time.

Bathtub of bucks

inflows of $$ increase the tub level, outflows of $$ decrease the tub level

Now, imagine that instead of water, the tub holds metaphorical dollars. The tub can be thought of as an account, a set of funds, ‘budget number’, set of budget numbers, or similar. The inflows then are one or more sources for adding dollars to that tub with a mechanism or set of constraints that determines the rate of flow into the tub. Similarly, there is a mechanism for setting how much flows out of the tub (spending or investing).

City and institutional spending

Cities and institutions have multiple sources of inflows, most of which they probably don’t control. Those inflows have independent characteristics from each other as well as some interdependencies with each other. The main takeaway is that the city or institution probably does not control a whole lot regarding what’s coming in.

The spending from the top tub can go to multiple places, themselves other tubs. From the top bathtub, most organizations make decisions between operational dollars (running things) and capital dollars (buying or building big things).

splitting between operational $$ (running things) & capital $$ (buying or building things)

IT & cybersec resources & spending

From the operational dollars tub, some funding goes to IT operations, some goes to cybersecurity operations (eg CISO’s office), and other funding goes to many other traditional and important areas such as HR, finance, policy/law enforcement, and others.

Operational dollars, in turn, gets disbursed across multiple other operational tubs

In the interest of keeping the diagram simpler for our discussion, we won’t include capital spending or non-IT/cybersec spending in subsequent diagrams.

IT systems services and cybersecurity services

Funds from the IT operational bathtub are used to resource the management of various IT systems and sub-systems in the institution or city. This includes both on-premise systems as well as cloud-based systems. Examples include enterprise resource management (ERP) systems, institutional learning/training systems, calendaring and email systems, and others.

Systems that have known performance expectations and implementation precedents (either themselves or peer implementations) can provide the basis for a fairly reasonable calculation to be made on required staffing and funding support requirements.

Similarly, the city/institutional department/organization providing information security services  (usually the CISO’s office) also has a set of well-managed services that are planned for and delivered. Examples of these information security services might include: education and outreach, incident management capability, privacy policy guidance, intelligence analysis, and others. The CISO’s office will work to develop services and capabilities based on the IT systems that the city or institution is operating, known and evolving threats and vulnerabilities, existing risk levels, and others.

resourcing planned and reasonably well-managed systems and services


The trouble with unplanned, under-managed, and unmanaged systems

Managing and identifying management support resources can be challenging enough with known systems. Challenges and institutional risk quickly becomes exacerbated though when unplanned or weakly planned systems are added. For example, after the budget/planning cycle, an influential person or group may decide that the city or institution “must have” System X. And then later someone else with influence might insist on (unplanned) System Y.

When these unplanned or under-planned systems are added, several deleterious things can happen:

  • the unplanned system drains from the IT operational funding tub in the forms of implementation staffing, management staffing, and support tools  
  • planned systems now no longer have their expected resources and they themselves can become under-managed in addition to the add-on system that is very likely also to be under-managed
  • institutional/city risk increases because unmanaged/under-managed systems increase likelihood of system comprise due to misconfiguration, mismanagement, lack of oversight, failure of (or lack of application of) controls
  • things get worse as the problem also transmits to a different bathtub, ie the information security services provider for the city or institution, eg the CISO’s office
  • when compromise occurs — particularly on systems that the CISO’s office could not plan for — the CISO’s office is now forced to work in a reactionary mode. This is expensive and pulls resources from planned cybersecurity services

unplanned, under-managed systems also transmit their problems to the CISO’s office in the form of increased likelihood of systems compromise

IoT Systems often fall into the unplanned, under-managed category

Several aspects of IoT Systems deployments can contribute to them having high risk of being weakly planned and under-managed systems —

  • lack of precedent for implementation & management
    • cities/institutions don’t have deep experience with these systems
      • true for all phases – systems selection, procurement, implementation, & management
    • few, if any, peer cities/institutions from which to learn for systems expected to last years or decades (sufficient time hasn’t gone by)
  • accountability and ownership unclear
    • IoT systems span many organizations within a city or institution
    • most organizations are not familiar or practiced at coordinating with each other in this role
  • acquisition path – IoT Systems can come into the institution through many non-traditional paths
    • these IoT Systems are rarely acquired by central IT
    • even if acquired through central IT, traditional systems vetting approaches not sufficient
  • no established vetting of IoT systems prior to purchase
    • performance expectations unknown or unclear (see ownership above)
  • the city or institutional department acquiring the system might not be the one supporting the system
  • Newness and rapid evolution IoT Systems makes them hard to discuss, categorize, and plan for

the newness, novelty, and rapid evolution of IoT Systems will continue to make very susceptible to being under-managed systems

Rapid evolution of IoT Systems vs glacial pace of institutional change

While there are no silver bullets or magic technologies (and we shouldn’t spend much time looking for them) to address these added risks that IoT Systems bring, there are things that we can do now, or at least begin now, that can positively impact our risk exposure as institutions and cities. While we’re interested in mitigating risks that we have now from IoT Systems, the impact of IoT systems in our cities and institutions in the future will be much higher.

Some things that can be done now include —

  • establish a set of criteria for your city’s or institution’s for IoT Systems
    • circulate and share this across the organization
    • one starting point is here
    • a related document from the Internet2 IoT task force on IoT risk  is here
  • identify IoT Systems ownership and accountability
    • require before acquisition
  • identify institutional language used to communicate traditional risk & incorporate that into IoT risk conversations, guidelines, and planning
  • consider an IoT Systems oversight group for your city or institution

Making broad changes, perception changes, and policy changes in cities and institutions is arduous work that takes time, leadership, political capital, and patience.  It is important that we begin now because this level of institutional change will likely take some time and the impact of not making the changes is increasing rapidly.


Who’s building your IoT data pipeline?

data pipelines require labor too

IoT data pipelines require labor too

There is a lot of excitement around IoT systems for companies, institutions, and governments that sense, aggregate, and publish useful, previously unknown information via dashboards, visualizations, and reporting.  In particular, there has been much focus on the IoT endpoints that sense energy and environmental quantities and qualities in a multitude of ways. Similarly, everybody and their brother has a dashboard to sell you. And while visualization tools are not yet as numerous, they too are growing in number and availability. What is easy to miss, though, in IoT sensor and reporting deployments is that pipeline of data collection, aggregation, processing, and reporting/visualization. How does the data get from all of those sensors and processes along the way so that it can be reported or visualized? This part can be more difficult than it initially appears.

Getting from the many Point A’s to Point B

While ‘big data’, analysis, and visualization have been around for a few years and continue to evolve, the new possibilities brought on by IoT sensing devices have been most recent news.  Getting an individual’s IoT data from the point of sensing to a dashboard or similar reporting tool is generally not an issue. This is because data is coming from only one (or a few) sensing points and will (in theory) only be used by that individual. However, for companies, institutions, and governments that seek to leverage IoT sensing and aggregating systems to bring about increased operating effectiveness and ultimately cost savings, this is not a trivial task. Reliably and continuously collecting data from hundreds, thousands, or more points, is not a slam dunk.

Companies and governments typically have pre-existing network and computing infrastructure systems. For these new IoT systems to work, the many sensing devices (IoT endpoints) need to be installed by competent and trusted professionals. Further, that data needs to be collected and aggregated by a system or device that knows how to talk to the endpoints. After that (possibly before), the data needs to be processed to handle sensing anomalies across the array of sensors and from there, the creation of operational/system health data and indicators is highly desirable so that the new IoT system can be monitored and maintained. Finally, data analysis and massaging is completed so that the data can be published in a dashboard, report, or visualization. The question is, who does this work? Who connects these dots and maintains that connection?

who supplies the labor to build the pipeline?

who supplies the labor to build the pipeline?

The supplier of the IoT endpoint devices won’t be the ones to build this data pipeline. Similarly, the provider for the visualization/reporting technology won’t build the pipeline. It probably will default to the company or government that is purchasing the new IoT technology to build and maintain that data pipeline. In turn, to meet the additional demand on effort, this means that the labor will need to be contracted out or diverted from internal resources, both of which incur costs, whether direct cost or in opportunity cost.

Patching IoT endpoint devices – Surprise! it probably won’t get done

Additional effects of implementing large numbers of IoT devices and maintaining the health of the same include:

  1. the requirement to patch the devices, or
  2. accept the risk of unpatched devices, or
  3. some hybrid of the two.

Unless the IoT endpoint vendor supplies the ability to automatically patch endpoint devices in a way that works on your network, those devices probably won’t get patched.  From a risk management point of view, probably the best approach is to identify the highest risk endpoint devices and try to keep those patched. But I suspect even that will be difficult to accomplish.

Also, as endpoint devices become increasingly complicated and have richer feature sets to remain competitive, they have increased ability to do more damage to your network and assets or those of others on the Internet. Any one of the above options increase cost to the organization and yet that cost typically goes unseen.

Labor leaks

Anticipating labor requirements along the IoT data pipeline is critical for IoT system success. However, this part is often not seen and leaks away. We tend to get caught up in the IoT devices at the beginning of the data pipeline and the fancy dashboards at the end and forget about the hard work of building a quality pipeline in the middle. In our defense, this is where the bulk of the marketing and sales efforts are — at each end of the pipeline. This effort of building a reliable, secure, and continuous data pipeline is a part of the socket concept that I mentioned in an earlier post, Systems in the Seam – Shortcomings in IoT systems implementation.

With rapidly evolving new sensing technologies and new ways to integrate and represent data, we are in an exciting time and have the potential to be more productive, profitable, safe, and efficient. However, it’s not all magical — the need for labor has not gone away. The requirement to connect the dots, the devices and points along the pipeline, is still there. If we don’t capture this component, our ROI from IoT systems investments will never be what we hoped it would be.


Good cybersecurity advice to SMB’s from California AG


Kamala Harris, Attorney General, California Department of Justice

Kamala Harris, Attorney General, California has posted some pretty good cybersecurity advice for small and medium sized businesses (SMB’s) in that state.

California has 3.5 million small businesses which represents 99% of all employers. The report states 98% of their SMB’s use wireless technology of some sort, 85% use smartphones, 67% using websites, 41% on Facebook, and 36% using LinkedIn.  I would speculate that other states, while not as large, probably have similar percentages of types of technology use.

The document covers threats such as social engineering scams, network attacks, physical attacks, and mobile attacks as threats to SMB’s in that state. Overviews of data protection and encryption, access control, incident response, and authentication mechanisms are also provided.


The core tenets espoused by the document are:

  1. Assume you’re a target
  2. Lead by example
  3. Map your data
  4. Encrypt your data
  5. Bank securely
  6. Defend yourself
  7. Educate employees
  8. Be password wise
  9. Operate securely
  10. Plan for the worst

This document does a great job of providing an overview of cybersecurity issues and initial effort prioritization for SMB’s. It would be great to see other States follow their lead.

Motivating adoption of cybersecurity frameworks

US-WhiteHouse-LogoThe Federal government is seeking to motivate businesses that operate our nation’s critical infrastructure systems to voluntarily adopt a Cybersecurity Framework currently under development by NIST (National Institute of Standards and Technology).  These systems include the electricity generation and distribution grid, transportation systems, and drinking water storage and distribution systems.  A preliminary draft is available now here and it will also be presented in two weeks at the University of Texas.

Roughly simultaneously, the Departments of Homeland Security, Treasury, and Commerce have been developing various options to try to provide incentives for companies to voluntarily adopt the Framework.  Per the White House Blog, there are eight core areas or approaches to incentives under consideration.

  1. Engage the insurance industry to develop a robust cybersecurity insurance market.  As discussed in an earlier post, this is not without it’s challenges.
  2. Require adoption of the Framework for consideration of Federal grants related to critical infrastructure or include as a weighted criteria as a part of the grant evaluation process.  This seems reasonable to me, though it only incentivizes those companies applying for Federal grants (but maybe that’s most companies?)
  3. Expedite government service provision for various programs based upon adoption of the voluntary Framework adoption.  Again, seems logical, though this one seems a short step away from changing the ‘voluntary’ part of the Framework adoption.
  4. Somehow reduce liability exposure of companies that adopt the Framework.  Per the White House Blog, this could include reduced tort liability, limited indemnity, higher burdens of proof, and/or the creation of Federal legal privilege that preempts State disclosure requirements. If one were a cynic, that last one could sound like buying a loop hole.  This whole core area of modifying liability seems to be to be pretty tough to manage, particularly to manage transparently and equitably.
  5. The White House Blog says that “Streamlining Regulations” would be another motivator for participating companies.  I don’t get this one.  I don’t understand how the government could “streamline regulations” for one company but not for another.  Sounds to me like interpret the law one way for one company and another way for another company.
  6. Provide optional public recognition for participating companies.  This one seems like a good idea.  Sort of a Good Housekeeping Seal of Approval, Better Business Bureau endorsement, or similar to Joint Commission on Accreditation of Healthcare Organizations endorsement for hospitals.
  7. Companies in regulated industries such as utilities could be offered some sort of rate recovery contingent upon adoption.  This seems reasonable logically, but I would imagine a bear to implement and manage (which is kind of a theme for many of these).
  8. The White House Blog says that “cybersecurity research” is an incentive.  This one I don’t get either. How does identifying weak spots in the Framework and encouraging research in those weak areas motivate Framework adoption? I mean it’s a good thing to do, but how does that make any one particular company want to participate.

While these are proposals for incentives for critical infrastructure companies, I’m wondering if some of these can serve as a model for SMB’s for adoption of cybersecurity standards for SMBs. Adjusting cyber insurance premiums based on participation would seem to be an obvious approach. However, as has been discussed previously, a mature cyber insurance market does not yet exist and it’s not a slam dunk that one will evolve sufficiently fast to address this need.  For SMB’s seeking government grants, to include SBIR (Small Business Innovation Research) grants, compliance with an SMB cybersecurity framework would seem to be a no brainer. Also, optional public recognition for compliance with an SMB cybersecurity framework would seem to be a practical approach.

What would motivate you as an SMB to adopt an established Framework?