David Autor | Expertise, AI, and the Work of the Future

Sarah Devany:

Welcome to our final 2024 event from the Emerging Tech Economic Research Network, also known as EERN. I’m Sarah Devany and I serve as the first vice president here at the Federal Reserve Bank of San Francisco. I also serve on the executive leadership advisory Group for EERN.

EERN seminars are opportunities for academics, researchers, and technologists who are studying the economic impacts of emerging technologies to exchange ideas, learn about research from different sectors, and share insights more broadly with those who are interested in these topics. For us at the Fed, they also provide important insights on how developments from emerging technologies can potentially impact productivity and the labor market, both of which are critical to the Fed’s dual mandate of stable prices and full employment.

We’ve had such a great response to our EERN speaker seminars where we’ve heard from noted economists on how generative artificial intelligence, or gen AI, is shaping the future economy. I urge you to explore all of our past event recordings on our San Francisco Fed website. Today’s final installment of this year’s series will explore the labor market impacts of AI. To help us explore these possible futures, we’ll hear from David Autor, professor of economics at MIT. Professor Autor will present his research on technology and job market dynamics. Following his presentation, professor Autor will discuss the research with our host moderator Huiyu Li, co-head of EERN and research advisor at the San Francisco Fed. I’m really looking forward to this event. I’ll turn things over to you, Huiyu.

Huiyu Li:

Thank you, Sarah. Please join me in welcoming Professor Autor from MIT to our EERN seminar, today. I’m very excited to hear from Professor Autor. He has followed an unusual career path and instead of the usual academic escalator for economists, he dropped out of college, learned how to code, worked as a software programmer, then went back to college and volunteered for many years before going to graduate school. In fact, one of his volunteering jobs was here in San Francisco where he taught computing skills to disadvantaged youth. So I think these personal experiences motivated much of his research on the labor market impact of emerging technologies. So without further ado, let’s hear from Professor Autor on the labor market impacts of AI.

David Autor:

Thank you so much. It’s an honor to be here. I’m only sorry I’m not with you in person, but I’ll do my best to appear animated. So I want to speak about expertise, artificial intelligence, and the work of the future. Let me begin with just a rhetorical question. What is the difference between these two jobs? Air traffic controller and crossing guard? If you think about it for a minute, these are basically the same job. The job is to prevent collisions between planes and buses or planes and other planes, or children and motor vehicles on the way to school. And yet crossing guards make less than a quarter of what air traffic controllers do. Why is that? It’s not really because of social value. If we had to spend $130,000 a year to prevent our children from colliding with cars on the way to school, we would surely spend that money.

The difference is expertise. To become a crossing guard in the United States requires really no training or certification. Any able-bodied adult can potentially do that job. To be an air traffic controller requires years of air traffic control school and then thousands of hours of apprenticeship to master the skills. And so one way to say this is: If we suddenly ran short of crossing guards, we could get the air traffic controllers to do that work. But if we suddenly ran short of air traffic controllers, we could not get crossing guards to do that work. So expertise is the theme of my talk. So expertise means domain specific knowledge or competency to accomplish a particular goal, right? So it could be coding an app, baking a loaf of bread, examining a patient’s vital signs, remodeling a kitchen, et cetera. Expertise is what makes labor valuable in industrialized countries where physically demanding work is less and less prevalent.

But not all expertise is valuable. To be valuable, to command a substantial market wage, you need two things: One is the goal that it enables, or must itself have, market value. So it’s got to be data science, not card tricks. The other is the expertise must be scarce. To paraphrase the character “Syndrome” from the movie “The Incredibles,” when everyone is special, no one is special. When everyone is expert, no one is expert. Expertise intrinsically corresponds to this notion of something that some people know how to do and not everyone knows how to do. Why do we care? We care because non-expert work pays poorly. So waiters, cleaners, janitors, security guards, school bus drivers, daycare teachers, crossing guards, right? Some of these are life and death activities. These are not unimportant jobs and yet they pay poorly because most people can do them. They don’t require substantial expertise. And so the supply of potential workers to those occupations is extremely elastic because it is extremely available and that tends to keep wages low, except when labor markets are extremely tight.

Now I want to emphasize how important the role is of labor in economic activity. And you might think for example, that in the richest countries in the world where we have the most technology, the most capital, the most equipment, the most roads, the most buildings and so on, that labor would be a relatively small part of economic activity. But the opposite is true in North America, six out of every ten dollars of economic activity is first paid to workers before it goes into the rest of the economy. The world average of that is 54%. In parts of Asia or in North Africa it’s as low as 45%. So it’s actually kind of a miracle of the rich world that even as we have created all this automation, all this technology and built all this capital and infrastructure, simultaneously we’ve made our labor more valuable.

And I don’t just mean we’ve raised our wages because we’re more productive. I mean labor share of economic activity is higher, right? More money goes to labor, not less. That means less money goes to capital and everything else. So that’s remarkable. Something we should appreciate and value. We actually have a shared concern for labor remaining valuable because it’s how most of us make our incomes. And if our labor wasn’t valuable, it would be a very different kind of a society.

Now let me talk about the value of labor or really the value of expertise in two different eras. I want to talk about it in the computer era and I want to talk about it in the era of artificial intelligence. Let me start with the computer era, just a quick review for you. So the computer era arrives, we can say in the 1970s, 1980s. And what it does is… computers are very relevant for carrying out what you would call tasks, routine tasks, tasks that can be codified, systematized, turn into scripts and executed by computers that have no judgment, no problem solving capability, no common sense, no ability to improvise, but they can carry out steps really, really effectively.

And this made them very relevant for doing work in offices and in factories. So either production and operative positions or clerical, administrative support and sales. This actually skilled work. In fact, a lot of this work really grew very rapidly during the era of the second industrial revolution in factories and offices and employed more than half of all workers. And these are workers with high school degrees, they had skills, they had experience, and what they did was sophisticated. It required mastering rules, learning how to use tools, whether those tools were welding machines on assembly lines or whether they were calculators and adding machines and filing systems and typewriters. And computerization had the effect of hollowing out a lot of that work. Why? Because although that was skilled work for humans, it was extremely susceptible to what computers were good at, which was following rules.

And from 1950 to 2010, according to work by Bill Nordhaus, a Nobel winning economist at Yale, the cost of computing fell a trillion fold. In other words, a set of calculations that would’ve cost you a trillion dollars in 1950 cost you a dollar in 2010. So it automated and it left this kind of barbell shape. On the one hand here, I’ve organized occupations from lowest paid to highest paid, right? So here, personal services, then these middle skill office, clerical, administrative support, production jobs, and then professional, technical, and managerial job. And so as computerization progressed, it kind of led to this something of a barbell phenomenon with increasing poundage on either end of these bars. Now that wasn’t all computers. There’s many other things going on. I don’t mean to oversimplify, but it was an important contributor. And the problem with that is it meant that people who were pushed out of the middle or next generations pushed out of the middle and didn’t go into these higher paying occupations, found themselves doing food service, cleaning, security, entertainment, recreation, crossing guards.

Again, why is that a problem? There’s nothing wrong with that work. It’s valuable work. It just doesn’t pay well because it’s not expert work, because many, many people can do it. So this period was one in which we saw a huge amount of wage polarization. People who were in professional technical and managerial jobs, college graduates and people with post-college degrees, they saw pretty strong earnings growth, especially people with post-college educations from kind of 1980 forward. But people who had less than a four-year college degree, and especially people who were high school educated only or didn’t have a high school degree, especially if they were male, they saw really profound earnings declines as the value of work in production and offices declined. And many more people found themselves doing that type of crossing guard work that was not well remunerated. Women did better, but still we saw this incredible fanning out.

They might say, well, I showed you this barbell, what’s going in at the end of the bars? Why wasn’t computerization relevant to the green bars, the red bars? Why only the blue bars? Well, the answer comes down to what some call Polanyi’s Paradox. So Polanyi was a contemporary philosopher who made the following observation, which is that we know more than we can tell, many of the things that we do, we understand tacitly but not explicitly, right? So if you’re trying to think of how do I tell a funny joke or come up with a novel hypothesis or make a compelling argument, or how do I slice a tomato in my kitchen or ride a bicycle? We don’t actually know the formal rules and procedures. You don’t go up to a blackboard and learn how to ride a bike. You learn these tacitly through experience. You learn them from immersion.

So when you first rode a bicycle, presumably your parents didn’t teach you about gyroscopic physics. They just put you on the bike on a gentle hill and gave you a push and said, good luck. But that’s how a lot of learning occurs. Well, why is that relevant to computers? Well, it meant historically we couldn’t computerize the tasks that we didn’t explicitly understand. If we didn’t know the formal rules and procedures, we couldn’t get computers to do those activities, because we couldn’t write the program. So we could write the program for a lot of operative and production jobs. We could write the program for a lot of office clerical, administrative, and sales jobs, but we actually couldn’t write the program for people who were doing medical work or legal work or analytic work where there’s so open-ended, nor can we write the program for people who are historically, who are driving vehicles, who are cleaning rooms, who are serving as crossing guards. Again, because those require a type of knowledge that we’ve not really formalized and still are struggling to formalize. And that of course is the frontier of robotics.

So that’s where we’ve been and that really describes in very, very broad brush strokes the last 50 years. But we’re not in that era anymore. We are in the area of artificial intelligence. And what I mean by that is that computers can do all kinds of things that don’t fit that description of following rules. So we now have machines that can autonomously learn from unstructured text. They can do tasks that we would think of as creative if it were done by people, like for example making these illustrations. They can even discover and invent things that we don’t understand.

So, in October of 2024, many people became aware of the so-called protein folding problem. So the protein folding problem is that for many, many proteins we would know or scientists will know the amino acid sequence, but they wouldn’t know the actual structure of the protein, how it folded as a function of all the magnetic charges. And a protein might look like this kind of party streamer, for example. And this is such a hard problem that figuring out the folding structure of a single protein could be a million dollar investment, could be the work of a single PhD thesis. And it was such an important problem because this is critical to structural biology. It’s critical to medicine. The way medicines interact with your body is not just a function of what amino acids they contain, but actually what is their physical structure? How do they fold? And so a person named John Moult used to run a competition, still does from 1994 forward, called the Critical Assessment of Protein Structure Prediction (CASP). And people would enter, as in 2020, 215 teams would enter, scientists. Moult would give them the amino acid sequence of a set of proteins. They would have a few days to then come back and give their best prediction of what the folding structure was of those proteins. Presumably they didn’t know the answers already.

So, this was a problem that people were making incremental progress on. And then in 2020, Google, well, actually it wasn’t Google, a company called DeepMind, now owned by Google, entered a product called Alpha Fold. And Alpha Fold was a competitor in CASP. And as Science Magazine wrote in 2020, “The Game Has changed. AI Triumphs at Protein foldings” – “Artificial intelligence has solved one of biology’s grand challenges.” This machine blew people out of the water. It could solve this problem or at least do a very good job of it, at a rate of accuracy and speed that humans could not touch. John Moult, the co-founder of CASP, said, “this is a 50-year-old problem. I never thought I’d see this in my lifetime.” And I mentioned October 2024 because in October 2024, the Nobel Prize in chemistry was awarded to Demis Hassabis and John Jumper of DeepMind for this piece of software.

This is an amazing accomplishment and it’s not something that could have been done with traditional computing. Because why? Because AI can solve, can learn tacitly like we do. It can without rules, without programming, it can make inferences about what is going on, what to do next, what is likely to happen. And so AI learns a lot. It’s not the same. But it learns as people do from experience, from learning from a set of experiences and associations. And that enables a whole bunch of capabilities that were not previously feasible, like solving the protein folding problem.

So how does that relate to labor markets? So I want to come now back to expertise. And the first remark I want to make is, look, AI is a tool. It’s a tool like a chainsaw or a calculator. And many tools are valuable to us. They actually make our expertise more valuable. So if you were a doctor, you would certainly not want to be without your stethoscope and other instruments. If you’re a roofer, you certainly wouldn’t want to do your job without a pneumatic hammer. If you’re an air traffic controller and you don’t have a two-way radio and a radar and a GPS, you are blind to your job. You are a person standing in a room staring at the sky. You can’t actually do your work. So in many, many ways, tools can augment the value of human expertise. In fact, much of our expertise is bound up in the tools that we use. If you took away my computer, I could not do my research in any finite amount of time. And for many of us, that’s true.

So the potential good scenario is that AI could actually enable people to extend the value of their expertise. So when we think about AI and expertise, the good scenario is one in which we take our expertise and we can use it more broadly. We can accomplish more or we can do harder problems or enter domains that we previously could not enter because we have better tools for doing so, right? Just like the stethoscope or the pneumatic hammer or the GPS and the radar.

But that’s not always true. Not all tools are friendly to expertise, right? So from 1894 forward, if you wanted to become a London black taxi driver, you had to spend about three years memorizing all the streets and highways and byways and underpasses of London. It’s an incredibly arduous task. The brain scans actually show how people’s brains change as a result of this act. And then when phone-based routing was introduced, it made that knowledge economically irrelevant, it stranded it. It’s not that we didn’t need to know how to get around London, but Waze knows that. And not only does it know all those highways and roots and byways and underpasses, but it also knows how much traffic is on them in any given moment, which is not something you could ever learn from memorization. So it’s quite possible for AI or any tool to augment this value, some expertise, and make other expertise superfluous.

So now let me talk about how this can work in practice. So let’s start off by observing that AI is already woven into daily life, right? So if your smartwatch asks you, “have you taken a fall?” Or your modern day car tries to steer you back into a lane on the highway, or your writing application offers to assist you in completing your writing task, these are all cases where AI is basically observing what you do, trying to make predictions about what you want to happen, and then giving you advice and input to complete that activity. So a lot of what AI does at present, when you encounter it, is it makes predictions about what you want based on what it’s seeing.

So how does that interact with human expertise? Well, there’s both a promise and a peril. There’s upside and there’s downside. Let me start with some downsides. And I give you two examples. One of them is really pretty dramatic and I think and relevant. The other one is much more contemporary and I think has very close parallel. So let’s see where this goes.

So in June of 2009, Air France flight 447 took off from Rio de Janeiro on its way to Paris. Out over the Atlantic at about 35,000 feet, the Pitot tubes froze over. Those are just the speed sensors on the aircraft and having them freeze is not that uncommon an event, but it means that the aircraft cannot detect its own velocity without the Pitot tubes. The autopilot that was navigating the plane at the time partially disengaged at that point, as it is supposed to do, right? That’s a failure mode when it can’t sense its speed, it can’t pilot the plane. It hands over to the pilot. The plane rolled. In the confusion that followed, the crew stalled the plane. So what does it mean to stall the plane? To stall the plane doesn’t mean like stalling a car where the engine shut off or your speed drops to zero. It means that the angle of attack of the wing relative to the air it’s facing is so steep that the Bernoulli effect, that gives the air wing lift, breaks down. You don’t any longer have laminar flow of air over the wing, and so the Bernoulli effect that creates suction effectively above the wing stops creating suction. And that means the plane loses the ability to maintain altitude.

During the three and a half minutes between when the autopilot turned off partway and the plane collided with the Atlantic ocean, the crew did not understand what was going on. In fact, the voice recorders have been recovered and you hear the autopilot saying over the loudspeakers in the cabin, “stall, stall, stall.” And then you hear the crew saying, “what’s happening? I don’t understand what’s happening. This cannot be happening.” And only in the final 30 seconds did the crew realize the plane had stalled. But at that point it was actually, it was too late to correct course. So the conclusion of the crash report for this aircraft was that the crew lacked the experience on the characteristics of high altitude manual flying. They had lost the feel for what they were doing. Now I want to say that’s not always true. Think of Captain Sully Sullenberger who landed at 737 in the Hudson River after a bird strike, took out the jets on that plane. So this is not necessarily the modal case, but it is an important example of the negative interaction between human expertise and machine expertise.

Let me give you another example that’s much more contemporary and now I hope will seem more familiar. So this is a scan from a patient’s lungs and it’s being read by a machine called CheXpert, chest x-ray expert. And what this machine does is it looks at the scan and it makes a prediction about what it sees. It says edema is quite likely. That’s the red area here. Pneumonia is very likely, that’s the pleual effusions down here. And it shares that information with the radiologist. Now, this is a really good application of AI because when you’re reading x-rays, there are no exact right and wrong answers. You have to recognize the patterns. You become familiar with what looks like an edema, what doesn’t, and you look at someone’s lungs and you say, well, it looks okay, go home. Or you say, let’s go for further tests, right?

So this is something that people develop judgment on over the course of thousands of hours of practice. And you can think that AI would be really good at this because you can train it. You don’t have to tell the rules. You simply say, here’s scans, here’s how they’ve been diagnosed. Now you infer what’s going on. And of course, an x-ray radiologist, an AI, can look at many more tens of thousands of scans than any human could look at in their lifetime. Turns out this product is pretty good. So using just the information in the scans, it does as well as about 64% of radiologists. So really quite good. So you might think that the combination of radiologists and CheXpert would be even better, but in fact that’s not the case. So researched by my colleague, Nihal Agrawal and Tobias Salz and their co-authors finds just the opposite, that actually radiologists using CheXpert do systematically worse than radiologists just looking at scans on their own.

And you’d say, well, how could that be true? It’s a good tool. Why did they do worse? Well, the problem is what they call a correlated uncertainty, which is that the x-ray, the software and the doctor are looking at the same scan. So they’re both going to tend to be confident or not confident at the same time. So let’s say they’re not confident, the doctor’s not sure, the CheXpert’s not sure. And what is it by not sure, it says, you know, pretty likely or not likely or not certain, right? It tells you how confident it is. Well, what does the doctor do in that situation? Well, typically the doctor will defer to the machine. On the other hand, let’s say they’re both pretty confident and yet they differ. That doesn’t happen that often. The doctor will tend to override the machine.

Well, it turns out neither of those is the right instinct. When there’s a lot of uncertainty, actually people are much better at extrapolating out of sample. They have much better judgment. When there’s a lot of certainty, the doctor should at least ask themselves, well, look, this thing has looked at more scans than I will see in 10 lifetimes, how am I so sure it’s wrong? And so because the doctors have the wrong instincts, they actually do worse with this tool than without it. Now, what you should be taking away from this is not that this is a bad tool or not that there’s no way for this to work, but this is a bad design, right? These doctors never had a chance to develop judgment. It’s like people flying with the autopilot and never having the chance to practice with the autopilot off. How would they know when they should trust it and when they should not trust it? Because they never had an opportunity to practice. This is not like they sat in a simulator with x-rays and work with this machine.

In fact, the only reason this is known is because this experiment was done. These doctors thought they were reading new x-rays when in fact they were reading old X-rays where the answer was known. And then my colleagues could determine how well did they do reading these x-rays using CheXpert or not using CheXpert. Turned out they did worse, and now we know why. This is a solvable problem. It’s a design problem, and design problems are going to be a major issue for us going forward in figuring out how to use AI well. It’s a sophisticated tool. It has its own judgment. We have to interact with it with our judgment, and the combination could be better or worse depending on how we use it.

So, let me talk to you for a few minutes about some better cases about where expertise can augment human potential, human expertise – excuse me, where AI can now augment human expertise. So, this is from a well-known study by two students of ours at MIT, Shaked Noy and Whitney Zhang. And they ran an experiment in the early days of generative AI back in 2023. And they ran an experiment on Prolific, which is just a platform for hiring people to do tasks. And they got college graduates who do some writing for a living. And they asked them to do two tasks: One was to write a little marketing study, like a very brief document, and the other was to create some advertising copy. And then between the first and second tasks, they randomly chose half of the people in the experiment said, “hey, here’s this tool called ChatGPT 3.5, here’s a little bit of training on how to use it. If you want, you can use it in addition to Google and whatever else you use to do this writing.

So the black dash line is the people who were not offered ChatGPT. The green line is people who were offered it. And what you see is in terms of speed, the people who used chat GPT were much faster. Their time savings, they went from about 28 minutes to about 18 minutes to about a 40% time savings. So that’s impressive. It turns out also the quality of the work they did was somewhat better. So they hired another set, excuse me, of college graduates to evaluate the work on terms of brevity, clarity, precision, novelty, and the people using ChatGPT did about a half a standard deviation better. That’s good. That’s a bump of about more than another 15 percentage points, let’s say, relative to others.

But the main cool result is that if you look at the relationship between people’s score on their first task and their second, the dash line again in BlackLine is the people who didn’t use ChatGPT. And there’s a pretty steep slope here, it’s about 0.5. So if you did well the first time, you probably did pretty well the second time, and if you did badly, you probably did badly again. Not exactly, there’s some mean reversion. If you look at the people who used ChatGPT, you see this kind of flattening out of this curve. The least effective writers not using ChatGPT, were about as good as the median writer using ChatGPT. So you see this kind of compression of performance. It’s not that it makes everyone a great writer. There’s still a slope. And it’s not that it makes the best writers better, it does not, but it makes the less effective writers, more like the more effective writers, right? It’s a tool that compliments their expertise. So this is a really extraordinary case, and we’ve seen more like it.

But let me give you an even more recent example. This is actually a PhD student of ours at MIT, also named Aidan Toner-Rodgers. And he looked at the use of generative AI in material science. So material science is designing materials, metals, glasses, fabrics, glues that have desirable properties might be heat resistant, or they might be unbreakable, or they might be stretchable, or they might stick things together. And as you would expect, AI has become a big deal in material science. So if you look since 2023, there’s been more than 2000 material science publications that mentioned AI. And in PhD syllabi, more than a thousand.

Let me not spend a lot of time on how it works, but it does work. And the way it works is it proposes new molecules or new materials for sciences to evaluate and experiment with. It’s good at this. And so this paper asks two questions. One, how well does this work? And two, how does it work with people or how do people work with it? So the answer to the first question is, it works really well. So after this is adopted, the rate of new materials discovery increases by about 40%. The rate of initial patent filings increased by 30%. So patent, it takes years to get a patent approved. So patent filing is the right sort of approximate outcome. And then the rate of new product introduction or prototypes goes up by about 20%. So this turns out to be quite effective, scientists are more efficient. And moreover, the properties of materials that they produce are more closely aligned with the engineering targets and by some counts appear to be more creative in terms of the novelty of the designs.

Now you ask, how did people do with it? Well, this had a huge effect on how people spent their time. So prior to using this tool, people spent about 40% of their time generating ideas, another quarter on judgment, and another third on experimentation. Once the tool was in place, they spent only one eighth of their time on generating new ideas, one sixth of their time, 40% on judgment, and 44% on experimentation. So you can see that in the figure on the right, right? It massively changed the job. So what that meant really is that AI is proposing ideas and scientists are evaluating them and experimenting with them. You say, well, how did scientists do? Well, you can compare people, the average scientists and how effective they were. And effectively, one way you can measure that is how many of their experiments succeed. And if they’re doing this well, their first experiment should have a high success rate, and then you should go down the curve.

So you can see without AI, people had very high targeting and then they ran out of ideas pretty quickly. With AI, they had less good targeting, although more ideas on average. However, if you look at people who initially were at the bottom quartile of productivity and the top quartile, you’d see the people at the bottom quartile, they weren’t very good at judging initially. But once AI came into place, they’re effectively drawing at random. There’s no relationship between what order they prioritize things and which ones turned out to be good. Whereas people who were very effective initially, their targeting still wasn’t as good with AI, but they had a very long tail. So they able to supply the judgment to use this tool well, and that’s kind of the bottom line of the study, that although the tool was effective, it was only effective with about a third of scientists. And what distinguished those scientists is they had already deep-seated judgment about what types of materials would work well. And so again, the answer, the takeaway as with CheXpert, is not that it’s a bad tool or it’s a good tool, but that it requires a skillset that people have to either have or acquire, and that requires thoughtful design.

So, let me say, these are just examples, and we have lots of cases now, but they all point in different directions. Some cases we see this leveling up in terms of writing, customer support, in terms of software. But in some cases, we will see displacement, we’ll see amplification of superstars as in material science, we’ll see expertise being diluted as in those taxi drivers.

So let me spend the last five more minutes talking about a related concern. Here I am talking about expertise, what makes human labor valuable. And you may be asking, well, will we need any labor? Aren’t we just going to run out of work? Who cares what type of jobs, will we have any jobs at all?

And if you have that concern, you are in distinguished company. I’m not going to say good company, but distinguished company. Here’s Elon Musk last year saying, “there will come a point where no job is needed,” and the DOGE Commission will ensure that that comes sooner rather than later. Here’s Geoffrey Hinton, who also won the Nobel Prize this year for his work on neural networks, which is foundational to AI saying, “get a job as a plumber.” So we have two visionaries here. One envisions a future with no work, another envisions a future where everyone’s a plumber. Neither of them sounds very interesting to me as a labor economist.

So I’m going to tell you two reasons why I think these predictions are incorrect. One of them is pretty mundane, and one of them is a little more profound. So let me start with the mundane, concrete one. As many people will be aware or will have directly experienced over the last five years, we’re not running out of jobs, we’re running out of workers. In fact, throughout the industrialized world, the working age population is falling in numerical terms. So Korea is expected to lose 40% of its working age population by 2060, China is expected to lose 30% of it. The average in the OECD is 10%.

Now, you might say, well, what’s wrong with having a smaller population? Is that a problem? Well, it is a problem because when a population contracts, it’s not that it contracts evenly across all the age ranges. It means you run out of young people, still have many older folks who have earned retirement, expected to be cared for and will have to do that. And so when a population shrinks, it also ages a lot, and that creates a severe economic problem. So all evidence is that we’re running scarce on workers, and I don’t expect that to change. Now, I want to emphasize, that scarcity is artificial. There’s many, many people throughout the world who would be thrilled to come to any of these countries and do work in those countries. So it’s really immigration restrictions are a policy choice. And so labor scarcity is a policy choice. But there’s no evidence that that policy choice is turning in favor of immigration, so I expect that we will face this challenge for a while.

But now let me turn to a more deep-seated reason. When many people think about what new technology is for, they think about what will we automate? What is the next thing that machines will do that we use to do ourselves? And there’s a lot of automation in the world, so that’s reasonable. But most innovations, and what makes the most innovative is not that they automate work, it’s that they instantiate new capabilities. So before machine powered flight was mastered by humanity, we didn’t fly at all, right? This didn’t automate the way we used to fly. We simply didn’t fly before we had flight, mechanical flight. Or the scanning electron microscope did not automate the way we used to look at subatomic particles. We simply couldn’t see them, right?

If you went back to ancient Greece and automated all the stuff they were doing, you would not have modern San Francisco. You would have ancient Greece without horses, it wouldn’t have flight, it wouldn’t have telecommunications, it wouldn’t have penicillin. Most of what is transformative about technology is not that it makes the old stuff better, cheaper, faster, although it does that, it’s that it creates new capabilities that were just not conceivable, not possible without those tools. And AI will be similar. So when we think about what will AI do to us, we should be asking the question, what do we want it to do for us? What is the purposes to which we want to put it? Because we have a lot of choice of this, right? We can use technologies in a variety of ways.

Just to give you a dramatic example, right? We have two very constructive, well, sorry, two very powerful uses for controlled nuclear fission. One is to create warheads. Another is to create power plants. North Korea has an impressive arsenal of warheads. It has no nuclear power plants whatsoever. Japan, which is the only country against which an offensive nuclear weapon has ever been used, has as far as we know, no nuclear weapons, but it has dozens of nuclear power plants. That’s a choice about the prioritization of technology. It’s not decided by the technology itself. It’s decided by the societies that use it.

Similarly, we can use AI for censorship and surveillance. It’s a really good tool for that, and it wouldn’t be possible to surveil and sensor at the scale that some countries do without AI as a tool. So AI can be used for that. But that’s a choice, that’s not intrinsic to AI. We could instead use it to assist with elder care, perform robotic surgery. We could provide real-time information to construction workers and maintenance workers and contractors to improve the type of work they do, or enable them to do more advanced work. We could make learning more immersive, more engaging, more accessible, less expensive. These are all possible uses of AI. So when we think about the AI future, we should not think in terms of what it inevitably will do, but what choices we’re going to make about what we want it to do.

So, the questions we should be asking is, will we use AI to empower and extend expertise to extend human judgment into domains where we’re not as effective as we could be, or will we use it primarily to automate work, displace workers. Both things are feasible and both will occur. I want to be clear. But we have a choice about where we focus our energy. If we want to extend human expertise, how do we make that happen? This is not an easy problem to solve. This is an opportunity, it’s a necessity. We can’t just… saying do that, doesn’t tell us how to do it. And it’s also a collective choice. It’s going to be affected by government, by industry, by universities, by labor unions. All of them are going to have a hand in shaping this. They may not all have equal hands, but we should all recognize it’s a choice.

So let me end with the following observation… the case that my whirlpool washing machine has more processing power than the Apollo guidance computer that landed the first humans on the moon in 1966. And for that reason, I’m able to start my washing machine from anywhere on the planet, on surface earth, not up from the moon, but on the planet. And that’s of course a completely useless, worthless technology to me. I don’t need to start my laundry room, my washing machine from anywhere other than in my laundry room, since I’m going to have to be at the machine to take my laundry in and out.

And more profoundly, my washing machine is never going to the moon. And so what makes the technology on the right a huge innovation, and what makes the technology on the left a kind of pointless gimmick, is not the processing power, but the human imagination that goes into it. The capability to do something new with a new tool. So when we think about what will we do with AI, we should be looking for moonshots, not just replicating what we can already do, better, cheaper, faster, but finding new capabilities that we can unlock, new opportunities that we can discover. And that’s where a lot of the value is going to come from. Okay, let me stop there.

Huiyu Li:

Alright, thank you very much, David, for that very engaging talk. So I will start with a few questions of my own since I’m the moderator, and then I’ll switch to pre-submitted questions. And we may also have some questions from the audience. So the first question I would like to discuss with you is on I guess timeframe. So in what timeframe do you think we’ll see some of these labor market impacts of AI and also the productivity effects?

David Autor:

Sure. I think we’re seeing already some of these impacts. Certainly, I think the slow down in hiring of software engineers. There’s some evidence of decline in certain types of managerial work. And then there’s different types of work that are growing, including different types of customer support work. But, I think we’re in the very, very early days, not in terms of the technologies’ capabilities, but our knowledge of how to use it well. This is a different tool. It has different strengths and weaknesses. And the very first reaction, the first instinct anyone has with a new tool is to say, oh, what can it do better, cheaper, faster than my old tool? But usually it’s not the case that it’s actually simply better at that old thing. It may even be worse. Artificial intelligence is unreliable with facts and numbers, not like your traditional computer, but what it can do instead or that you didn’t think of.

So, we’ve seen this in the past, in the history of electrification. It took people a long time to realize that having electric motors didn’t just allow you to rip out steam engines and coal engines. It allowed you to put tiny little motors in very specific places to do much more precise work. And that took decades for people to realize. So I think if there were no advances in AI for the next 20 years except for maybe getting more power efficient, we would still find 20 years of innovation to do with it. But at this point, and of course there will be substantial advances, but at this point I think we’re still groping around. And I think many, many applications of AI will turn out to be a bust. And many of the companies that are being invested in will turn out to be a bust. That doesn’t mean there won’t be successful ones, but you can sort of think we’re in the kind of pets.com age of AI where there’s a lot of good ideas and bad ideas all mixed together, and people are having a hard time telling them apart.

Huiyu Li:

I think your studies in the past have shown that new tasks are being created. So even though automation takes away some tasks, new things are being created, do you think that timeframe will be faster for generative AI than in the past compared to say automation?

David Autor:

Yeah, great question. So let me sort of fill in a little bit of the picture that you asked about. So when I talked about new expertise, new expertise often means knowledge of how to do something new. So in recent work I did with Anna Salomons and Caroline Chin, and Bryan Seegmiller, co-authors of mine, we looked at the growth of new work, work that required new forms of expertise. And we found that almost 60% of work that people were doing in 2020, that form of expertise was not present in 1940, right? Whether you’re a pediatric oncologist or you’re a computer security engineer or a windmill technician, or also if you’re a marital therapist or a meeting planner, these are all forms of specialties that were not present. And many of them are bound up in actually new tools. So certainly when you think about the oncologists or the drone pilots or the AI specialists, they’re bound up in new technologies. Not all of them. Some of them just have to do with changing tastes, changing income, changing demographics.

It’s a bit of a mystery where new work comes from. We really have trouble predicting it, and we have trouble predicting how fast it will occur. And I think that’s the concern, is that I do think new things will occur, but they may take a while to get here and we don’t know what they’ll look like. And so when I worry about the consequences of AI for the labor market, I don’t worry about us running out jobs, but I do worry about the elimination of expert work, and of machines, like those taxi cab drivers, that take valuable expertise that people have spent years developing and making it so cheap that it actually has no economic relevance to those people.

Now, let me be clear, consumers benefit from that, right? Taxis are cheaper, but most of us are consumers and world workers and our livelihoods depend upon our expertise. And so when that landscape changes really fast, it really creates problems. And so I think one should be optimistic about the potential for using AI well and simultaneously realistic about the real risks. And there will be job displacement. And even if we create new work just as fast as we displace it, the people for whom we create new work will be different from the people who are displaced from old work. So even then, if it’s a pure wash, it will still create serious consequences, not all of them positive.

Huiyu Li:

Great. I want to follow up on that point about the people who get the new work are different from the people who are displaced. Is retraining or rescaling somehow will help us retrain the workers who are displaced so that they’re not affected negatively so much. Would it be less painful than, say, manufacturing workers who lost their jobs and find them really hard to find another position?

David Autor:

Yeah, I mean, so the US does not have a proud history of success in adult education programs. There are other countries that do this much better. It’s actually difficult for adults to retrain. Most people do not want to go back to school. They didn’t love it the first time. They’re not going to like it much better the second time now that they have kids and car payments and mortgages and so on. And so it’s challenging to do this well. It doesn’t mean there are no examples. And I hope it is the case that we can use technology to do it better, to make more things more like flight simulators and less like blackboards, for example. But I think we need to also recognize that we need to support workers who are displaced. And there’s actually really interesting research about… the Obama administration did a pilot of what was called wage insurance for people who were trade displaced. Wage insurance just means that for the first couple of years, if you take a lower paying job effort, after being displaced by trade, part of the income is refunded to you or paid for, the government makes up part of the difference.

Now, this didn’t get people into higher paying jobs, but did get them back into the labor market more quickly, and that’s valuable. So I think we’re going to need a variety of tools to support people. And I will say the US spends one tenth as much as a share of GDP on worker, what are called active labor market programs, as does for example, Denmark, which is really, really good at this. So I think we need to recognize how hard this problem is and recognize we’re going to make big investments and not all of them are going to work out, but it really matters for people’s welfare. Losing a job is pretty much up there with losing a marriage in terms of how miserable it makes people and how damaging, traumatic it is over the long term.

Huiyu Li:

Yes, I’m still procrastinating on learning how to write prompts for gen AI, for research. I guess we all are learning it through just day-to-day interaction with gen AI, but to use it as an expert requires, I guess more training. Yeah.

David Autor:

Well, I mean it requires a lot of practice. I mean, I really think many of these things, it’s an opaque tool. You’re going to have to develop a feel for it just like you do for your car or the knife you use for making food or many tools that we use. We develop judgment and it’s tacit. It’s not that we sit down at a blackboard and learn it. We learn it through experience, and people do get better at this as they use AI.

Huiyu Li:

I want to switch topics a little bit, ask about potential bottlenecks. It was interesting that you mentioned there was a decline in the amount of time these material scientists spend on idea generation. I’m thinking that current AI is learning from data that’s already there or ideas that have been generated. And if people stop or slow down, how much they spend on creating ideas, will we run out of ideas for AI to learn from? Or is it AI will create enough ideas that we have this virtuous cycle that AI will create ideas, we’ll learn from them, and growth will continue? I wonder what are your views on that?

David Autor:

Yeah, so there is a lot of… there’s people I think the term they use is the “data desert” or something that we run out of new data, then the AI creates its own data and it’s kind of like trying to power your car from the fumes it spits out the back. I don’t really take those scenarios that seriously. I think that AI is going to be a tool that people will use for discovery. I mean, I went to a talk last night by Terence Tao, who’s a famous mathematician at UCLA, and he was talking about the use of AI and mathematics. He says, look, it’s going to change the job of mathematicians, but it’s going to change the type of problems we can work on and the scale of them and even the way we work. Basically, mathematicians work in tiny teams because just people can’t have enough in their heads at one time to do the work. You cannot have a big math team. And that used to be true for the way that software is developed. But now software is developed in large teams because we have better tools for managing that. And mathematics can be the same way.

So I don’t think we’re going to run out of ideas. I think that this will be, I don’t want to say an accelerator, but it’ll be something that allows people to prototype ideas faster, figure out which ones work and advance them more quickly. So I think it’s an incredibly powerful tool for simulation and for poking around. And even if you use gen AI for writing, what I’ve discovered is it’s not that good at writing, but it’s actually pretty good at editing and it’s valuable to interact with it. It’ll make suggestions. Some of them are good, some are bad. But if you have good judgment about writing, it can actually help you improve it.

Huiyu Li:

Yes. Like a sounding board.

David Autor:

Yeah.

Huiyu Li:

And before I move to the pre-submitted questions, I want to ask about what are some moonshots that you think are important. Personally, I think that medicine, increasing quality of life, reducing the cost of education, those are all important things. I just want to pick your brain on what do you think are important moonshots that maybe we should think of?

David Autor:

Yeah, so you hit the two I think are most important. So between medicine and education, healthcare and education is more than 20% of GDP, and at least half of that is public money. So we have really a stake and we also have a case, it’s also the case that healthcare is rationed, education is rationed, and we’re not very efficient at it. And we know this. We’re not getting much better, much faster. We have better technologies, but our actual ability to increase the number of people who are treated effectively, quickly, and cheaply is not very good. And education hasn’t changed a lot in decades. So those are areas where we could use these tools better. And using them would better, like in medicine I think especially, it wouldn’t just be a matter of making things better, cheaper, faster, but also change the way we use human expertise.

And if I’d had time, I would’ve talked about the example of nurse practitioners: Medical professionals, highly educated, but do a lot of tasks that used to be done only by medical doctors who had five additional years of education. Now, this has nothing to do with AI. These are very capable people and they had to fight crazy actually to get their own scope of practice. But now they use these tools that enable them to diagnose, to treat, to prescribe more effectively, more safely. And I think AI will extend and can extend the type, the range of tasks, the range of medical tasks that others can accomplish. So using AI well in medicine wouldn’t just mean getting better, cheaper care, but also using labor better, enabling more people to do more diagnostic and treatment tasks without the bottleneck always of the highest educated, most expensive person in the building. Not that those people aren’t necessary, but they’re not necessary for everything.

Similarly, in education, I don’t see us getting rid of teachers. I certainly hope not. But I think we could give teachers better tools to customize learning for students as needed and support students differently. So I think these are two areas, and in neither of these cases, are we going to run out of workers, but we are going to democratize access to these vital services.

Huiyu Li:

Thank you. I will switch to pre-submitted questions and questions from the audience. One question we just got from the audience is about forecasting. We work for the central bank, so of course we want to know what’s going to happen. So do you see gen AI being a potential tool that’s used for forecasting?

David Autor:

You know, I don’t know. So first of all, I’m not a macroeconomist. I don’t do any forecasting and I’m pretty cautious about it. But I think the thing is, it’s very hard to forecast things that are not common. It’s easy to forecast things that have, where there’s a lot of outcome. You can’t forecast recessions because they don’t happen that often. And there’s always some baseline probability, but when it’ll actually occur, I don’t know. And it’s often tied to some very idiosyncratic event. Go ahead.

Huiyu Li:

No, no, continue.

David Autor:

Okay. Yeah, and so there’s a good book by Arvind Narayanan at Princeton and his co-author Sayash Kapoor, called “AI Snake Oil.” And it’s actually not as negative of a book about AI as the title would suggest, but they do talk about what problems it’s applicable for and when it’s not. And it’s really good for things where they have a well-determined central tendency and where there’s a lot of cases. I don’t think for kind-of one-off events like forecasting wars, forecasting recessions, forecasting financial crises. I doubt it’ll ever be very good for that.

Huiyu Li:

Okay, thank you. We had many questions from both the audience as well as pre-submitted questions that involve the impact of gen AI and equality and inclusion. So there is the income distribution side, but also just in general, are there inclusion concerns from gen AI?

David Autor:

Okay, good. So I think there’s inclusion concerns, but also inclusion upsides as well. For example, work by Nick Bloom at Stanford shows that the employment rate of people with physical work limitations has risen a lot since the pandemic. And an important reason for that is telecommuting. And in fact, many machine tools can help people, people who are dyslexic can write using gen AI. And screen readers can now, I don’t know if you’ve seen this recently, but when my phone gets a text and I’m driving and the texting contains a picture, it describes the picture to me. It says this is a picture of potted plants against a white wall, which I don’t know why someone texted that to me, but they did yesterday. And so anyway, I think there’s a lot of possibilities for actually including people who would actually be excluded because these tools can compensate or help people overcome specific limitations.

I think there is certainly a possibility for increasing inequality, but also for decreasing inequality. If we use AI to enable more people who are not the elite experts to do work valuable work in medicine, in education, in law and design, and contracting and repair, like those nurse practitioners that I was speaking of, for example. That would be a really good scenario and it would create more competition for the professions. That’s okay. The professions have had a good 50 years and it would be nice, actually, I would be happy to be in a world where medicine, where doctors were a little bit less scarce because there were more people who could do expert medical care. And so I think both are possible and I don’t know what the central tendency will be.

I do know we are living in a period, most people probably are not aware of this, where inequality actually has come down a lot in the last five years, come down since the pandemic, and mostly because wages have risen at the bottom of the distribution. Now this is inequality of earnings. It doesn’t include capital income. Knowing what the stock market has done, I’m sure there’s a lot of more inequality of capital income than there used to be. But most people get most of their earnings from the labor market. So I view this as a very positive development and it partly, by the way, has to do with labor scarcity with our really tight labor markets, which I was talking about earlier.

Huiyu Li:

Thank you. We actually get a lot of students dialing in for our talks, and we usually get the question about how do you think AI will affect education in particular, if they’re going into the job market soon, how should they prepare themselves for something that could be very different?

David Autor:

Yeah, so I think that a quintessential human skill that’s going to remain really important is the kind, I don’t even like to use this term, but the kind of metacognition, the ability to synthesize information and make good decisions. And this is actually a really difficult skill. And this decision about what should we do? How should we lead this team? How should we prioritize what is a promising idea and a not promising idea? And the world has changed enormously. When I went to college, facts were actually pretty scarce, and you had to go the library to find them, and you had to look through book after book, and you had to take notes, with your hand on paper and so on. And now we live in a world where there’s abundant information, but a lot of it actually isn’t factual and it’s not reliable. And the hard problem is not finding information, it’s determining what’s reliable and synthesizing that and making good decisions.

And that’s where a ton of human judgment comes in. And that’s what we actually learn over time. Whenever you enter any profession, whether that’s being a plumber, an electrician, a lawyer, a doctor, a scholar, you spend hundreds, thousands of hours immersed in it to develop that judgment. And so I think that skill for synthesizing information and using tools well will become more, not less important. But it does mean you need to use AI well, because many times humans are good at identifying what the problem is, but then coming up with a solution actually may be analytically quite hard. If you have good tools, you can do a better job of this. So often the human expertise comes in saying, well, what is the actual problem here? And then what are the variety of solutions? And now we’ve got to use our tools to sort of simulate, to experiment, to produce. So I think there’s a lot of opportunity and certainly not everyone needs to be an AI engineer to benefit from that. In fact, there’ll be a very small number of AI engineers and most of us will be AI users.

Huiyu Li:

Thank you. Another set of topics we got are related to I guess, regulation of AI. And maybe this touches on the designing aspect that if we’re thinking about AI as a design question rather than a prediction question, who are the designers?

David Autor:

And I think this is actually, this is something my colleague Daren Acemoglu and Simon Johnson, who won the Nobel Prize, also in October, a lot of people won the Nobel Prize in October, more than any other month. Talk about a lot. What is the vision that people have when they’re engineering? And I do think there’s a lot of vision that’s driven by the notion of replacing human capabilities and making machines that replicate what people do. But I actually think we already have a lot of human capabilities and we have a lot of people who can do what people do. And so the more important question we should be asking is, what can we do that we couldn’t do without these tools? And I think that pushes design in a different direction. It’s still hard to answer that question in the abstract. So I like to think of it very concretely is what does this allow a person who is not part of the 40% of workers who have a college degree, what additional work does it enable them to do? Could their skills and expertise be more valuably applied with better tools? And that would be, in my mind, a really great use of artificial intelligence.

Huiyu Li:

Wow. Yeah. That’s a really great way to end it. So finally, on a more personal note, can you share with us what kind of AI tools do you use professionally or personally?

David Autor:

Sure. I mean, so I would say I use a lot of machine learning for turning text into data for my research. As a personal tool, I use it some for coding, even when I’m writing, creating presentations in LaTeX, which some will know is a very awkward and archaic but beautiful tool. It creates beautiful looking presentations after an infinite amount of work. AI is really good for that. But also I do increasingly use it for writing. I found it’s not good for first copy writing, but I found it’s really good for interacting with.

And so I spend dozens and dozens of hours every semester writing recommendation letters, and I can’t have gen AI write recommendation letters for me. In fact, when I’ve tried it makes stuff up and says things about my students that just aren’t even true. They sound great, they didn’t happen. But I have found that once I write the letter, I can give it to Claude or to ChatGBT and say, what do you think? And it can say, well, this paragraph would go here. This could use a detail. This is not a strong attitude. And it makes suggestions that help me read the letter slightly differently. And so I feel like that dialogue improves it. I don’t usually accept in whole cloth all these suggestions, but I do interact with that. I use my judgment and for me it’s to produce a better product.

So I think that’s valuable and I expect to use more of it. It’s kind of a resolution of mine to start using it to work with theoretical models as well. Some of it is pretty good at math. But again, I think the greatest danger with AI, as a worker, not in the world, is to sort of get out over your skis, to do something you don’t understand what you’re doing and you may or not have the right answer and you wouldn’t know. So it’s complimentary to having judgment and expertise. So it’s useful to extend what you know, but it’s not useful as a substitute for knowing nothing. I think bearing that in mind helps you figure out when you should rely on it and when you should steer clear.

Huiyu Li:

Yeah. Artificial intelligence does not replace human intelligence.

David Autor:

That’s right. It doesn’t make everyone an expert.

Huiyu Li:

Yes, yes. No. Thank you so much, David, for all the time you’ve given us. I’ve learned a lot from your talk. Thank you very much.

David Autor:

Thank you so much. It was really a pleasure. Thank you.