The pandemic has served to catalyze and accelerate seminal changes.  In this three-part article series, we will highlight some of the important themes and concepts that we believe will shape, and are shaping, the Future of Work in the post-COVID workplace.

In Part I, we assessed the pandemic’s influence in redefining where and how work gets done. And we discuss the implications across domains ranging from Real Estate and Commerce to Socializing and Education.

Part II (below) addresses on how the nature of work itself is evolving in a world of remote collaboration, hyper-specialization, and human-computer symbiosis. A useful prism to evaluate Artificial Intelligence software applications likely to gain traction, we will illustrate examples from the Software Development industry, Molecular Biology, Healthcare, as well as Prediction Markets.

In Part III, we will introduce the Pandemic Response Co-Lab, an initiative we are working on with MIT to mobilize innovators, communities, businesses, and others to develop actionable AI-enabled solutions to real problems post-COVID. To that end, we will highlight a few introductory examples of products and services that stand to benefit the workplace and workforce of the future.


Part II:  The Evolving Character of Work

The next question that arises is, in a world where people collaborate remotely, what kind of work will we actually do?  We believe that many changes will be dramatically accelerated by the bullet train of the COVID-19 pandemic.

We should be thinking less about people or computers and more about people and computers.  Less about how computers are going to take away jobs from people and more about what people and computers can do together that could never be done before. One useful way of thinking about these combinations of people and computers is as Superminds—groups of both people and computers, acting together in ways that are intelligent.

One of the fundamental questions we can ask ourselves, as we think about what work may look like in the post-COVID world, is: ‘How can people and computers be connected so that, collectively, they act more intelligently than any person, group or computer has ever done before?’

That’s a big hard, complicated question. But one thing that can help us answer it is thinking about or understanding the difference between two kinds of intelligence.

Specialized vs. Generalized Intelligence

The first is specialized intelligence, the ability to achieve specific goals in specific environments. The other is general intelligence, the ability to achieve a wide range of goals in a wide range different environments.

Even the most advanced computers in the world today have only specialized intelligence.  For instance, the IBM Watson computer program that beat the best human players of the game show Jeopardy—that program couldn’t even play tic-tac-toe, much less chess.  It was very specialized for the particular task of playing Jeopardy.

At the same time, all humans have more general intelligence than the most advanced computers in the world today. Even a normal five-year-old child, for instance, can carry on a sensible conversation about a much wider range of topics than today’s most advanced AI programs.

Now, one obvious question is how soon will this change? When will we have human level, general Artificial Intelligence? That’s a question people have been asking ever since the beginning of the field of Artificial Intelligence. For that entire time, an average answer people have given to the question is we should have human-level AI about 20 years in the future. In other words, human- level AI has been 20 years away for the last 60 years.

Now, is it theoretically possible that this time the prediction could be true? Yes, theoretically possible. But we think we should be very skeptical of anyone who confidently predicts we’ll have human-level AI in the next few decades. In our view, we’re likely to have that someday. But that’s likely to be many, many decades in the future. And part of what that means is that in the meantime all uses of computers will involve people.

From ‘Humans in the Loop’ to ‘Computers in the Group’

One way people talk about this issue today is to say that we need to have “humans in the loop.” When they say that, they’re often thinking about one person, one computer. But we think a more useful way of thinking about this is to start with the human groups that have accomplished almost everything we humans have ever done and then add computers to those groups.

Then, we can use the specialized intelligence of the computers to do the things they can do better than people, like arithmetic and certain kinds of pattern recognition. And we can use the general intelligence of the people to do everything else.  One way of summarizing this is to say that we need to move from thinking about ‘humans in the loop’ to ‘computers in the group.’

Let’s look at some examples of what it might look like to have computers in the group in the post-COVID world. One thing that is likely to become much more common is that when tasks can be done from anywhere on the planet, lots of new kinds of jobs will become possible.


One kind of new job category is that of hyperspecialized jobs. When you can have global economies of scale for doing things, it will become more common for people to specialize in very narrow tasks and be among the best in the world at doing these narrow tasks.

And others from all over the world can then avail that specialist’s skills. For instance, there might be people who specialize only in answering arcane questions from accountants all about what kinds of business entertainment expenses are deductible and which aren’t.  There might be other people who become hyperspecialists in multiple areas. One might spend the morning, for instance, evaluating the feasibility of pieces of Apple’s strategic plan, the afternoon estimating the probability that specific individuals are planning terrorist attacks in Yemen. Maybe they would end the day predicting the outcomes of local elections in Singapore.

Let’s look at a few examples of the kinds of work we are likely to see more and more of in the post-COVID reality: TopCoder, FoldIt, and the Human Diagnosis Project.


An example of what a future world of hyperspecialization might look like comes from a software development platform called TopCoder, which is now part of Wipro.  TopCoder is a community of over a million and a half freelance software developers from all over the world. Each software development project is divided into lots of small tasks, each of which is quite a bit smaller than the tasks performed in a typical software development process. The freelance developers compete to do these tasks.

The key point here is that different developers are able to specialize in very specialized tasks. For instance, there might be one developer who specializes only in doing user interface software for Apple iPhones, somebody else who specializes only in building a particular kind of database software on Amazon Web Services, and so on. The problem is thus split up into discrete modular parts, each of which is assigned to members of this distributed set of hyperspecialists.

Incidentally, software development is not at all unusual in this regard. There are many types of tasks where lots of different pieces can be done better via a distributed network of specialists dispersed all around the world.


Another example is a system called Foldit. It’s essentially a citizen science project for folding protein molecules. It turns out that one of the key questions in a lot of biology and biotechnology and pharmaceutical development problems is what particular configuration a protein molecule will fold into.

What happens scientifically is that it folds into the lowest energy configuration. And it turns out it’s pretty easy for computers to estimate the energy amount of a particular configuration. But there are trillions of possible configurations to be explored to find out what would be the lowest energy configuration.

Computers aren’t particularly good at figuring out which direction to explore those combinations. But some people who have a particular kind of visual aptitude are really good at that. The Foldit system creates online games that people can play that involve folding these molecules and trying to figure out how to fold them in ways that will have the lowest possible energy.

As it turns out, there are some people who have a particular kind of visual aptitude that makes them very good at doing this task. Those people have gravitated to the Foldit community. And they’ve been able to do some amazing things with this community. For instance, they once uncovered in only three weeks the structure of an enzyme related to AIDS that had eluded scientists for over 15 years.

Hyperspecialization + Artificial Intelligence:  The Human Diagnosis Project

Another example is that Human Diagnosis Project—a system that lets doctors, nurses, and other medical clinicians get help on their difficult patients and cases. The way it works is they put in information about those cases—for example, the patient’s symptoms, lab test results, etc. and then they ask for advice from other people anywhere in the world.

For instance, a doctor at Mass General Hospital here in Boston might get advice from doctors at Stanford Medical Center. Or more interestingly, a nurse in a remote African village hundreds of miles away from the nearest doctor might be able to get advice from anyone anywhere in the world.

What they find when they use this system is that getting more opinions leads to more accurate diagnoses and better clinical care. They also find that when they use the system in this way, they build up knowledge- based cases. And they’ve used Artificial Intelligence and Machine Learning to look for patterns in those cases.

What they found in doing that is that even with the relatively limited number of cases they have so far, that AI programs are able to be almost—not quite, but almost—as accurate as individual human physicians in diagnosing the cases.  And that’s with only a limited number of cases. What is particularly interesting here is what will happen when systems like this have seen millions of cases, far more than any human doctor could ever see in a lifetime.

When that happens, it’s quite likely that these AI systems will be able to give accurate diagnoses for many kinds of diseases. In fact, it’s likely that the AI will see diseases that we humans have never even noticed before.

That’s obviously a result of the AI capabilities of computers. But this, again, is another example of how you can use a combination of hyperconnectivity and Artificial Intelligence to do many different kinds of tasks, not just medical, as in this case, but lots of other things.


The Good Judgment Project at the University of Pennsylvania is part of a project funded by the US intelligence community. The goal here is to predict various geopolitical events like whether Brexit will be finalized by the end of this year.

One way of doing that would be to take people who are experts in these questions and ask them what they think. What the Good Judgment Project did was almost the opposite of that. They said that anyone with access to the internet is welcome to make these predictions.  They let people make predictions until the predictions either came true or didn’t.

And then they took the people whose predictions came true, the people who were in the top 2%, based on the accuracy of their predictions, they put those people in groups of about a dozen people each to compare notes. And then they used some statistical techniques for combining the predictions from different people in these groups. They called the people in these groups “superforecasters.”

What they found is that there are certain people who are apparently pretty good at taking publicly available information, mostly from the internet, analyzing it logically, and converting that into probabilistic predictions about when these different events would occur.

Now, it seems that they did a good job of it. These people did better than other people in general. But how do we judge the overall effectiveness of the group? It’s not exactly obvious what to compare them to.

One intriguing suggestion comes from someone who was part of the US intelligence community and also knew about the superforecasters project. That person was quoted in the Washington Post as saying that the superforecasters performed about 30% better in terms of accuracy than the average for US Intelligence community analysts who can read intercepts and had access to other kinds of top-secret data.

In other words, when you train, select, and combine the best forecasters from a more or less random online crowd of part-time workers, you get results that are substantially better than those from the multi-billion-dollar apparatus of the entire US intelligence community.

That’s a pretty interesting result. What could you do if you did better than just random online people? What if you gave these people special training and so forth?  We think lots of companies and governments would be very happy to have predictions about things that matter to them that were as accurate as this.

And we think that in the increasingly online world that the COVID pandemic is leading us to, it may be possible for many more people to make a living from doing hyperspecialized tasks like this.

Prediction Markets

Another way of making predictions is via something called prediction markets. It’s kind of like betting, but actually more like trading futures on a futures exchange. In this situation, participants buy and sell predictions about possible future events.

For instance, if you think that the given product is likely to sell between 1,500 and 1,600 units in September, you could buy shares of that prediction. And at the end of September, when the results are known, if you’re right and it is in that range, you’d get $1 for every share you own. And if you’re wrong, you get nothing for those shares.

Now, it turns out that the prices between 0 and 100 cents reflect the probability, the average probability estimates of the people participating in a market like this.

It also turns out that prediction markets have been found to be almost always as accurate and often more accurate than other forecasting methods like product sales, like focus groups and opinion polls and so forth for predicting things like product sales and elections.

For instance, a recent sample of prices in the prediction market for who will win the next US presidential election shows a 50% probability that Donald Trump will win and 44% for Joe Biden.  The historical evolution of prediction market prices shows that there was a brief period right after the pandemic became big news when Biden was a little bit ahead of Donald Trump. Now they’re still pretty close. But Trump is a little bit ahead of Biden.

Part of what’s interesting about this is that these people aren’t saying who they personally want to win. They’re predicting who they think actually will win. They have a financial incentive for doing that accurately.  And looking at these results is proving to be a way of getting a very accurate estimate of what’s likely to happen based on everything that’s publicly known today.

Human-Computer Combinations

While in this particular case the predictions are made by humans, an interesting question is what would happen if you combined people and computers to do this?

That’s a question we considered recently.  We picked what would likely be the next play in a football game. We showed people videos of a football game, stopped the video just before the next play, before the play began, and asked people to participate in a prediction market predicting whether the next play would be a run or a pass.

We also had some software agents trained using machine learning on one previous football game to make this prediction based on a simple information like what yard line the ball was on, and how many yards to the next first down.

In effect, we had some prediction markets where both people and computers participated in the same market. The simulation was set up so that a person wouldn’t know whether the last trade was made by another person or by a computer.

What happened was that we found that the combination of people and computers, as we had hoped, was both more accurate and more robust than either people alone or computers alone. This shows how prediction markets provide one way of combining the intelligence of people and computers in a way that can lead to surprisingly good results.

This is yet another example of how in the post-COVID world, there will likely be lots more work for people working online in various, unusual combinations with computers.


Stay tuned for Part III of this series, coming soon!