OPINION

Exposing the Software Supply Chain Problem.

Our manufacturing system is insecure.

The frail state of our physical supply chains has been laid before the eyes of the world, a revelation that started with the COVID-19 epidemic, but has continued apace after additional revelatory events. Most Americans realized it when they went to the toilet paper aisle to find that they would have to get creative in the bathroom. The panic ensued shortly after that.

The cost reduction and efficiency for the producers and manufacturers result in lower costs for consumers, a reason many of them have not questioned the practice.

For years, our total reliance on a system of just-in-time supply chains, in which goods are delivered right before they are needed, and excess inventory is kept to a minimum to reduce costs and increase efficiency, has worked out. The manufacturing process was originally developed by Toyota in the 1970s to help them build cars, but the system worked so well that it is now used in business all over the world of all sizes irrespective of the goods they manufacture. The cost reduction and efficiency for the producers and manufacturers result in lower costs for consumers, a reason many of them have not questioned the practice.

The fatal flaw in this global network of reliance, of course, is that when these supply chains are disrupted to a point where they cannot deliver goods on their vitally important razor-thin timeline—say, during a once in a century global catastrophe—people are not going to be able to get what they need when they need it. If the vulnerability were not apparent enough, on March 23rd, a 400-meter long container ship became wedged between the two adjacent banks of the Suez Canal, shining a light on the issue for all to see as the 12% of global trade that runs through the tiny waterway was now stopped abruptly. The blockage of the canal by the Evergreen ship supposedly cost four hundred million dollars per hour, an amount that is sure to be felt by countless organizations, who will pass on the cost to consumers. Unfortunately for just-in-time, the impact was not just monetary as the flaw in its system was starting to become glaringly apparent. Consumers and manufacturers alike had grown too comfortable with their supply chains, leading them to become insecure and costly when ignored.

Data center.

Data center.

MATTERS OF NATIONAL SECURITY

The effects of these supply-chain interruptions are starting to bear significant weight on more than just our commercial goods; they are impacting the national security of the United States. Consider the case of semiconductors, which the U.S. relies heavily upon Taiwan to produce. The strategically invaluable chips are used in a wide range of technology, both in consumer products and military weapons systems. The Taiwanese supply is used to satisfy 90% of the semiconductor needs of U.S. technology companies, and is so important to Washington that it is now looking to build strategic alliances with other nations and dedicate billions of dollars to start manufacturing at home. Imagine, then, if China were to invade Taiwan and seize the existing chip manufacturing capabilities of the island.

The opaqueness of their complexity grants the beneficiaries the excuse to be completely ignorant about their inner workings.

The complications of these global physical supply chains are now evident, but the problem does not end in the manufacturing industry. The software supply chain, which consumers and developers use every day without much thought, is similar to traditional supply chains, except it doesn’t deal in tactile goods. Instead, it focuses on delivering code and making sure that it is correct and integrated into applications and systems properly. It allows developers to churn out fantastic products and consumers to enjoy seamless productivity. This new ethereal type of supply chain has come under scrutiny as of late because of the recent cyberattack carried out by Russian actors.

The supply chain attack, which began sometime last Spring, though landing on people’s radar in the following Winter, continues to have ramifications today as companies and governments scour their networks for traces of intrusion. The incident involved an inconspicuous update to infrastructure monitoring software provided by SolarWinds, an IT management solutions firm. Unbeknownst to owners of the software, the update carried malware that would allow the threat actors who placed it there to carry out espionage and data theft in the networks of both private companies and government organizations who downloaded it.

The attack has been widely covered by the media and has provided the impetus for the Biden administration to impose retaliatory sanctions on 30 Russian individuals and six tech firms, and expel ten members of the Russian diplomatic mission. No doubt many people saw it on the news or read about it online without a clue to who SolarWinds is or what infrastructure monitoring software does for an organization. Despite widespread ignorance on the topic, the point of the matter was clear. A hostile foreign actor had hacked into software utilized by thousands of private and public organizations and planted malware that would compromise the networks and allow for future exploitation.

The SolarWinds hack was impactful enough to warrant mention in a recent annual threat assessment report by the Office of the Director of National Intelligence. Before calling out the Russian attack explicitly, it summarizes the issue by saying, “During the last decade, state-sponsored hackers have compromised software and IT service supply chains, helping them conduct operations—espionage, sabotage, and potentially prepositioning for warfighting.”

The trouble that comes with these supply chains is a result of the feature that makes them so successful. The opaqueness of their complexity grants the beneficiaries the excuse to be completely ignorant about their inner workings. It allows for customers to get goods quickly and efficiently without the need to raise as much as a curiosity into how the chain of events occurred. The shock of the shoppers when confronted with empty shelves is illustrative of this blindness.

A similar blindness occurs in the world of software, as nobody involved truly understands all the inner workings of a system. Users are constantly on applications that rely on hundreds of different software packages, or dependencies, to run properly. Even the developers of the applications would be hard-pressed to explain the complexity of all the dependencies they rely on. This seemingly innocent lack of clarity can result in a catastrophic situation—as seen in the SolarWinds breach.

The threat imposed by breaches like these is not flashy. The threat actor is not poisoning the water supply or shutting down the power grid. That is why it is imperative that people realize the true danger that comes from covert intelligence-gathering and espionage campaigns by foreign actors. Data and information are key to waging both cold and hot wars. If Russia, or China for that matter, can come into the networks of our government agencies or technology companies and gather up vital warfighting information or valuable intellectual property, they will be able to increase their ability to undermine the United States in the future. The software supply chain is one insecure avenue that our adversaries have been able to compromise, and, for the sake of national security, it must be addressed.

Circuit board.

Circuit board.

BUILDING TRUST IN THE SYSTEM

The proper solution to this new threat—that of insecure and frail supply chains—is not complete independence, but the strategy of blind reliance on external parties also cannot escape scrutiny. It would be impossible for a modern country to rely on their own economy to produce all that they needed. Similarly, modern IT infrastructures cannot be developed and maintained by their organization alone. The world is complex now and requires similarly complex answers.

Companies, especially those based in the United States, must take a hard look at how they integrate systems and software into their own operations.

There needs to be a paradigm shift in how consumers look at their products. The desire for everything to be made cheaper and quicker has fostered an ignorant assumption that the global network providing these benefits is without fault. Perhaps people would be willing to tolerate higher prices if it meant that the availability and security of the goods were ensured.

Similarly, manufacturers of these products must consider that this way of operating is not sustainable. There are accepted risks that come with using just-in-time supply chains and relying on the security of third-party systems and software. The problem now is that these risks and their costs may have been unknowingly approved.

Companies, especially those based in the United States, must take a hard look at how they integrate systems and software into their own operations. More stringent third-party risk assessments, security tools designed to protect applications at run time, and improving the trust verification mechanisms built into software updates are a few of the possible remedies. There is also the increasingly popular security architecture referred to as zero-trust. In this system, no person, device, or service is trusted to be non-malicious, leading to stringing conditions and controls being placed on actions and access requests made within a system.

Gone are the days of simple supply chains that relied on the goodwill and lack of error by the other entities involved. It is now apparent all around the globe that the longer this issue is ignored, the stronger and more devastating the impacts will become. Evaluating the trust placed in these complex systems will be necessary to avoid another empty shelf, blocked canal, or malicious software update.

Written By:

Caleb Larson is an information security engineer in the financial services industry and a member of InfraGard, a partnership between the FBI and private U.S. critical infrastructure organizations.