The ambiguity of intelligent algorithms: job killer or supporting assistant

This chapter argues that technological innovations in the field of ICT are driven by cost-efficiency but also by the necessity of providing an IT-based assistance for increasingly complex decision-making processes. It explores possible development paths depending on the extent to which humans lose or keep control over computer systems.

Increasingly, computers and autonomous robots will be able to assist – or take over – an increasing number of human activities. They will not only carry out complicated production steps completely by themselves, but will also perform thinking activities while delivering advisory and other services for knowledge workers and service providers. This creates new economic opportunities for Germany and its competitiveness against low-wage countries. In addition, developments in automation could be one approach to solving the skills shortage and reducing the burden of work on citizens. It may also change many existing job profiles and contribute to the emergence of new ones. Demand for suitably qualified personnel may produce opportunities for the job market.

However, it will also contribute to making a number of existing jobs superfluous. The elimination of low-skilled assembly-line jobs as a result of rationalisation could be followed by the elimination of supposedly skilled knowledge-worker jobs in the future [7, 9]. A number of professional fields in which comparatively high incomes are earned – such as medicine and law – involve many routine tasks, which increasingly may be performed by information technologies. It is expected that many jobs will be eliminated or change significantly with far-reaching challenges particularly for the middle class and for highly qualified professions: electronic medical expert systems in hospitals are already making diagnoses in hospitals, while on the world’s high-frequency exchanges, autonomous computer algorithms trade in millisecond cycles. Modern text-synthesis algorithms could very soon write news articles on sports events or compile financial news into reports that are barely distinguishable from texts by human authors. The middle class could be increasingly affected by these change processes due to loss of income or unemployment [10, 11].

The question arises as to the distribution of the “automation dividend” and the design of future social systems. It would be helpful here to have a scientific discourse about guiding principles that describe a balanced relationship between human and digital labour. It is important to systematically identify the extent to which striving for (profit) optimisation is ethically justifiable when using automation to raise productivity: How should society deal with the elimination of further jobs through rationalisation? What can be done to ensure that the current young generation is adequately prepared for digital competency pressure and changing job profiles?

Hence it is necessary to ask on the one hand what consequences such a development will have for the working world and society in the future, and on the other hand, how Germany will position itself in the international field of “algorithm providers” [12].

Driven by cost-efficiency and but also by the necessity of providing an IT-based assistance for increasingly complex decision-making processes, the possible development paths depend on the degree of autonomy of computer systems and the extent to which humans lose or keep control over these systems. The boundaries between conceivable scenarios are fluid, as outlined below in two extreme scenarios: extensive control and complete loss of control.

A development where humans have extensive control over computer systems and their applications would assume maximum transparency in the development of software systems and applications. System architectures should have a modular structure wherever possible, and intermediate products shall be clearly defined. Users will have operational control over systems that have an exclusively supporting function. For example, important decisions are prepared by computer systems, but the actual decision is always taken by a person following an evaluation and control step. With this development path, there is less of an impact on many knowledge-based jobs since people retain decision-making powers and responsibilities. Computers here have supporting functions for decision-making, and their interfaces in all areas of life and work are geared towards humans.

If computers assist people to a greater extent in knowledge-based work processes in the future, this will result in numerous opportunities. This assistance could have a positive effect on the growing desire for time sovereignty. However, if computers become more independent, this could mean that work processes and the people involved in them have to orientate themselves to technical systems. Yet surely an orientation to human requirements is desirable here [13]. People who know how to use computers and intelligent algorithms optimally for their work will have the biggest opportunities in the job market. Furthermore, automation opens up a variety of new opportunities to promote inclusion in the working environment. Individual work steps that cause problems can be performed by automated systems, and as a result, humans can retain their position in the value chain.

A development where humans completely lose control over computer systems and their applications would assume a massive spread of autonomous computer systems and big data replacing human intuition. Driven by cost-efficiency and the increasing inability of humans to take into account a large number of complexly linked facts and parameters in decision-making, computers would be used in ever more areas to make independent decisions that have direct or indirect consequences for humans. These systems would develop themselves independently, where humans are completely uninvolved (with IT specialists mainly ensuring that the systems do not fail.) An example for such a development is the increasing algo trading, where intelligent algorithms produce stock market reports, which in turn are analysed by other algorithms [14]. However, if assistance in the workplace goes so far that humans can increasingly be replaced by computer systems, job losses and sharp falls in income would be the consequence. It was estimated that up to 50 % of all jobs in the sales, office, transport and logistics sectors as well as administration and services could be eliminated [8].

The growing independence of software systems is accompanied by a continuous increase in technological complexity. Technical backgrounds and processes are ever less comprehensible to normal users because of their fast pace of development. Because of the large number of technical systems that surround humans, the vast majority of users are no longer expected to understand the underlying technology. Here humans are the direct driver of a loss of control which they tolerate. Even IT specialists usually only understand some portions of networked systems. The result of the interplay of complex feedback systems is becoming ever less comprehensible to individual people. Consequently, it is difficult to check whether decisions or solutions that are taken or calculated by computer systems are actually optimal.

If human correction is eliminated, it is possible that autonomous decisions by computer systems will be “neutral” and “impartial”, but on the other hand they could result in uncompromising harshness towards the people affected by them. When it comes to dealing with temporary payment difficulties on a home loan, for instance, algorithmic decisions may have a different outcome than the decision taken by a bank clerk, who is familiar with the borrower. Moreover, undetected system errors may have serious consequences such as incorrect calculations and wrong decisions. This creates the challenge of preserving transparency and control capabilities for humans in the case of autonomous decisions by computer systems and their innovations. Here, as in other areas too, questions of standardisation need to be answered.

The discussion about the dependence of humans on technical systems is not new. However, from today’s perspective, it is difficult to imagine the dimensions that a potential failure of networked technological systems might have in the future. It will be a key requirement in the future development of technology and innovations to consider the risks of technical failures or malfunctions. Related to this, the issue of liability should be addressed in case autonomous systems cause damage.

The German Foresight Process has portrayed in short stories⁸ how a “work colleague computer” that does so many jobs, works around the clock, never gets tired, and doesn’t make any careless mistakes, might influence the everyday life in 2030 and beyond. On the basis of these results and in fictional form, this chapter tries to illustrate challenges for the areas work, production, job and career. Potential implications of the “work colleague computer” and their possible ubiquity in everyday life are scrutinized in a narrative way, where the boundaries between conceivable scenarios are fluid. In this way, they may serve as a point of departure for further public discussions and dialogue.

A journalist sits in her home office, a cup of coffee in her hand. The computer has analysed yesterday evening’s football World Cup Final overnight, and written up a punchy draft article. News reports have been created automatically for some time now, and often they are even published on the online news site without any human intervention. It’s a great thing, as it means that the journalist has more time for editorial work that is more challenging and more interesting. The computer is also helpful for doing technical research. To start with, it automatically compiles relevant facts on almost any topic. Not only that, it prepares and organises its findings in a presentation. While the journalist was sleeping, the computer did the research for an upcoming interview with a professor of informatics about autonomous computer systems. It’s interesting to think about where it all might lead to. The journalist quickly skims through the prepared summary of the current state of development. The presentation ranges from automated business analysis to the evaluation of gigantic data volumes to detect all kinds of dangers, to the percentage of doctors who currently use intelligent diagnostic systems on an everyday basis, which frees up time to spend talking to their patients. This figure recently reached 85 %. The journalist is always surprised at how the significance of technology is changing in many occupations. His personal digital assistant, for example, has become indispensable as it performs all the secretarial duties: the computer routinely arranges appointments, places orders, answers calls and helps with management tasks. The computer informs the journalist that the virtual interview with the informatics professor will start in 5 min. Thanks to a holographic projection system, the journalist sits directly opposite the interviewee and saves a lot of time and money to be spent on alternative travel. However, since the journalist now only rarely needs to leave the house for interviews, personal contact to human beings has become seldom.

A labourer in production for an aircraft manufacturer has a mobile robot assistant by his side to help him with all the assembly work. The robot lifts heavy parts and sometimes even fits a tiny screw if it’s too fiddly for the labourer, who quickly learned to work with the robot assistant intuitively, but nevertheless took a training course to learn what the robot assistant is capable of. After a phase of getting to know each other that lasted several weeks, the robot learned to understand every spoken word, and even to recognise gestures and facial expressions. A few weeks ago, the robot rushed to the labourer’s aid when it saw that his face was contorted by pain. He had trapped his foot, but the robot quickly managed to free him. The aircraft manufacturer now uses more than 100 robot assistants in production just at this site, and has significantly expanded its global market position. The robot assistants have made a large contribution to this success, even though some production jobs have been lost. Cost-intensive work to rectify assembly defects is now rarely necessary. But robot assistants occasionally break down or report a malfunction. In an internal pilot project, a team of robots is working autonomously on individual assembly steps. Their perceptions, activities and communication with each other are fully monitored by collecting together all the data and information they generate in a cloud, so that it can be analysed someplace in the world. Initial concerns about data security were allayed. The aircraft manufacturer is one of the global pioneers in using robot assistants in production. The company’s management quickly recognised the potentials, and took the investment risk after a pilot phase. For some time now, the company has partnered with a mobile phone provider. They want to make smartphones in Germany and are interested in the aircraft manufacturer’s positive experiences with the robot assistants’ fine motor skills. Thanks to the use of robots, they will soon be ready to relocate production from Asia to Germany, and start making mobile devices for the consumer goods market in Germany again.

A family is looking forward to their summer holiday in Spain: their suitcases are packed and the driverless taxi is about to come and collect them. Their luggage is checked in automatically at the airport. The family won’t see their bags again until they get to their hotel room. Before the family leaves their house, they set the home automation controller to holiday mode. The controller switches lights on and off to make it look like someone’s at home, and even briefly opens windows safely located on the first floor to the tilt position. From the outside, you would hardly know that the house is unoccupied. Actually it’s not really vacant, as the home robot will water the flowers and tell them straight away if anything untoward happens in the house or garden.

The family enjoys the almost silent journey in the separate taxi lane. Journey times have fallen dramatically because of the taxi lane, and accidents have become a rarity. The city centre has been turned into a pilot area for testing autonomous transport systems, and is even closed off to private cars. Most passenger and goods transportation takes place via autonomous and driverless taxis, buses, trains and parcel drones. The journey ends almost directly in front of the check-in machine. This is possible because the luggage was transported separately in advance, which means it takes much less time to get out of the car. As they have some time to spare, the family goes into a fast-food restaurant, places their order at a display terminal, pays and receives an order slip. They can use this to log in at any table they like and the food is automatically delivered to the table. A service robot with a keep-warm function serves them. Although this is less personal than before, it does mean that the food is still piping hot when it arrives at the table. Many restaurant chains now use service robots like this, which enables them to stay open 24 h a day. However, during the flight, a flight attendant is available for the board service, not a robot.

While lifelong learning is omnipresent, it has become easier to study at a renowned international university, as nearly all universities worldwide offer affordable distance learning courses. In the market for creative production an open, global creative competition takes place. The best ideas count, and even for large companies it’s “in” to award assignments to less well-known creative thinkers (designers, etc.).

Following a downturn in sales in recent years, this has now led the established agencies to adopt new business models. These changes have been caused not least by new automation technologies, which make it possible for aspiring one-person design agencies to present clients with professional product designs. For example, a single freelance designer produces demonstrations of his designs overnight on a 3D printer. In the past, it used to take model-makers days to build product designs by hand.