Artificial intelligence facilitates the creation of models which are used to make predictions or informed decisions about future events. AI technology is currently used for facial recognition, speech recognition, language translation, personal recommendations for shopping, and lots of other practical applications.

In the near future, AI will be used to power autonomous vehicles, intelligent energy grids, smart personal assistants, etc. There is no doubt that AI will continue to make organizations more effective and efficient.

But there is one interesting problem with AI – biased data.

AI depends on data for it to work effectively. The humans inputting data knowingly or unknowingly pass their biases to the programs and that affects the output of the AI. Since AI depends on the data input from humans, it is not possible to have a neutral algorithm. A good example of this bias is when Microsoft was forced to pull down their twitter chatbot just a couple of hours after going live. The chatbot assimilated the opinions and conversations of its users and proceeded to post a series of sexist and racist tweets.

This is a perfect illustration of how an AI’s assimilation of new data in combination with contextual biases of a coder could result in some biased actions.

Transferable biases

Algorithms are typically trained for specific tasks or uses. The input data used can be biased thereby resulting in biased responses. The end user only interacts with the final learned model and they might never know that the AI was taught on biased data. Let’s take the example of an autonomous automobile that is being developed for the US audience. If the training data used for this car comes entirely from a single region or city (like the Ubers in Pittsburgh), the models that are made will undoubtedly be biased.

If all training data was from Pittsburgh, the self-driving car will learn and adapt to regional customs and norms of the region. This could lead to significant problems once the car is placed in the broader context of use without human supervision.

However, there are many biases that arise as a result of using an algorithm outside its intended context. For instance, in our earlier example of an autonomous car built for the American audience, the autonomy systems would suffer from transfer context bias if they were used in the UK, where driving happens on the left hand. This is the kind of bias that arises due to the use of an AI outside its intended context. A more subtle illustration is the transfer context bias that would arise in the translation of a healthcare algorithm from a research health facility to a rural clinic. The system will most certainly have an algorithmic bias because the two health facilities have different characteristics. For instance, the autonomous system might assume that the rural clinic has the same resources and that could result in a flawed allocation of resources. There is a thin line between training AI on bias and transfer context bias. For instance, if the autonomous vehicle in our example had been designed for worldwide driving, then what we would be having is a training data source of bias and not a transfer context bias.

Quality Verification

AI algorithms are often complex and it is this complexity that makes AI efficient and effective. For instance, NASA’s Remote Agent Experiment (RAX) is a complex system that ran a deep-space probe autonomously. This is, without a doubt, a very complex system and the benefits of this complexity are clear. But with such complexity also comes complex problems. A complex system is hard to understand and even harder to test. This is probably why RAX froze on its virgin mission due to a software deadlock problem.

On closer scrutiny, it was realized that the scheduling conflicts that caused the deadlock never arose during the testing phase. Hardware engineers easily solve these kinds of problems with hardware redundancy. In other words, when one hardware component fails for whatever reason, another one kicks in. unfortunately, redundancy cannot be used by software engineers to solve software problems.

Unlike hardware failure which is usually due to tear and wear or external damage, software failure is almost always as a result of latent design errors. This is why validation and quality verification must be taken seriously in the design of algorithms. The following techniques are used for quality verification:

Scenario based testing

Scenario based testing is the de facto software verification method since time immemorial. The system that is being tested is embedded into a test harness which will connect the outputs and inputs of the component and then run it through some test runs. An error is fired when the output received does not tally with what was expected. This kind of testing works just fine for simple systems but it may not be sufficient for complex AI systems. Complex AI systems have got too wide a range of situations so testing all of them may not be feasible.

Runtime monitoring

This refers to advanced methods for scrutinizing of artifacts in an executing program with the aim of detecting potential misbehavior or any other useful information. Software developers often add conditional clauses in their code to throw error messages when some error condition is fired. Run-time verification automates the strenuous process of manually reviewing logs. In more complex systems, algorithms can detect potential errors based on suspicious concurrent programming patterns.

Runtime monitoring doesn’t need too much computing resources and it, therefore, scales up nicely in complex systems. However, it can also result in false negatives, in the case of prediction of errors, e.g. when it flags potential errors which will not even occur. A good example of a runtime monitoring system is the Java path Explorer (JPaX).

Static analysis

This is the process of exploring the source code structure of a program without executing it. Techniques used include control and data-flow analysis, program slicing, and abstract interpretation. Static analysis can be applied to the source code quite early in the development stage. However, it might present interesting challenges when dealing with algorithms that are meant to result in huge numbers of false positives. Examples of static analysis tools are the PolySpace Verifier and CodeSurfer by GrammaTech. Static analysis tools are automatic but their use can be laborious, especially when it comes to understanding and correcting the identified errors.

Model checking

This is when a system is checked by exploring all possible states in order to ensure it satisfies a given property. Developed in the 1980s for testing communication protocols, model checking is now widely used in quality verification of most digital hardware. A mold checker typically searches the pathways of the system in order to identify any ways that a property can be violated. This means that the state should be tractable and finite. It may not be practical in scenarios like the space exploration programs where the number of states grow exponentially.

Theorem proving

A theorem prover works by building computer-supported proof through the logical induction over the program structure. A theorem prover can use the full mathematical logic power to test the quality of properties of a given design. Theorem provers are commonly used in the testing of NASA applications. The main demerit is they need lots of skill and effort from the user. The other quality verification methods are more automated and therefore preferred across different sectors.

Regulation for finance, insurance – and other regulated industries

Even though AI has often been viewed as a way to replace humans, the case cannot be so in the regulation and compliance field. When it comes to compliance it is not a matter of machine versus humans because both play a vital role. AI technology can be leveraged to boost efficiency while protecting the customer. The following are some of the practical scenarios of how AI technology is being used in finance and insurance.

  • Machine learning is used to assess credit quality for purposes of pricing and marketing insurance contracts and automating client interaction
  • Optimization of scarce capital by using back-testing models for the analysis of the market impact of taking certain trade size positions
  • Broker-dealers and hedge funds are using AI and machine learning to get signals for uncorrelated returns as well as optimized execution of trades
  • Various regulated institutions can use AI for surveillance, regulatory compliance, fraud detection and assessment of data quality.

The widespread application of AI can lead to new and unprecedented interconnectedness between institutions. For instance, previously unrelated data services might soon become a focal point for financial institutions and other types of organizations.

AI and machine language in the financial sector are relatively new and there is no international standard for purposes of regulation. But AI comes with some serious risks and there is a dire need for some standards to be set. For instance, some standard setters have analyzed the risks in algorithmic trading and have identified the pervasive nature of the markets as a point of concern.

Firms that develop algorithms should put in place a robust development process. This process should take into account all possible risks at all stages of the development cycle. This is important because it can help in the prevention of market abuse as well as the prevention of the algorithm causing any disorderly behavior on the market.

Accountability & Legal Exposure

There isn’t much regulation of AI because it is generally assumed that human users will use common sense when making final decisions. But this poses a huge challenge because humans often place too much trust in technology and thereby make poor decisions. For instance, incorrect GPS directions often result in accidents. In such scenarios, the software vendor might be sued on the basis of product liability and negligence.

Holding AI systems to account is important but the regulation of AI systems needs to be done carefully in order not to stifle the numerous AI benefits.

Explanation is arguably the most effective way of ensuring the accountability of AI systems. When the logic behind a decision is exposed, trust is not only increased but errors are avoided. The issue of “right to explanation” was recently debated by Europe’s General Data Protection Regulation (GDPR) opinion makers. It was realized that although limited explanation is required today, the future of AI will call for more explanation.

Nonetheless, there are some concerns that regulating the “right to explanation” might force developers to expose some trade secrets or it might limit the scope of innovations. Caution should, therefore, be taken to ensure this is enforced in a way that doesn’t curtail the progress of AI.