A ride. Hop in. Whats the main reaction you get from people riding in the car with you . Raj the typical reaction is one of anxiety and angst. Here, and occasionally fear and occasionally even panic attacks but then they basically watch the vehicle is able to drive, exactly stopping when it should stop. Ts actually riding fairly comfortably with a degree of comfort. We learn more about the move to selfdriving cars and professor raj rajkumar at the lab where his experimental car is kept and worked on. Dr. Raj rajkumar, whats your job here at Carnegie Mellon . Raj im a professor of electric engineering and robotics at Carnegie Mellon. Host howd you get into that . Raj i did my post graduate studies and got my masters at Carnegie Mellon. Then i left c. M. U. Joined i. B. M. Research for three years or so and i came back to c. M. U. Because i liked c. M. U. And pittsburgh that much. Host what kind of things do you work on . Raj i work on what are called embedded systems, the technical term. These are things that embed computers inside them but the research basically uses different devices. For example, your smart phone is a sophisticated device but you dont worry about the components inside there. Televisions, for example, et has components in inside it with you it is meant to be a elevision. That toaster that we use on a daily basis that cost 10 bucks its an embedded computer embedded inside a toaster. These are embedded systems that act smart but its a dedicated device of some other kind. Ive been working on embedded systems since my doctrine and it turns out vehicles that drive themselves are a classical example of embedded systems. So a car is a car that transports people and goods from point a to point b. With components embedded inside them. Host how did you get into the business of Autonomous Vehicles . Raj great question. Ive been working with General Motors, the carmaker since 2004. General motors has been working with the researchers in our Department Since the year 2000. I started applying my experts ease into embedded systems in the automobile. In 2006, the Research Advanced by the agency, the research arm of the u. S. Military announced a competition. The competition was the urban challenge. For vehicles that drive themselves, they need to drive for about 60 miles in fewer han six hours. With other selfdriving vehicles as well as human driven vehicles and following the same rules of the road that you and i have to follow on a daily basements g. M. Became our biggest sponsor. We had about 20 other sponsors but g. M. Was the biggest of them. Because i already had a strong working relationship with g. M. , i became became an integral part of the team that worked on the vehicle, which ended up winning the 1 million prize. When our team from Carnegie Mellon won the competition, g. M. , who was the biggest sponsor of the team said hey, our team actually sponsored the winning team and they said because its related to driving, it clearly has implications to the Consumer Market segment and they opened a second lab on campus focused exclusively on automated driving and ive been running that plan as well since itself launch. So our relationship with g. M. Continues to be extremely strong and very loyal. Host does g. M. Own the technology that you develop . Raj fmpt technology that we sponsor. Were grateful for their support. Is actually owned by caron darn university and we have Carnegie Mellon university and we have some Licensing Agreements with g. M. Host and we learned more about the experimental cadillac as we got ready to take it for a drive. What is this monitor thats up here in the top . Sensors. Six major thats one in the front of the car. Theres one in the bumper here. Right there. Radar, s the second thats the third on the other side of the bumper as well. Theres one behind the side back window and one on the other side. Host what are they read something raj theyre sending out multiple players of layers of beams and when the beams basically hit the on the, they bounce back and come back to we ransmitter and because know the speed of light, we can estimate how far that is. Because there are multiple beams scanning as well, you can get a profile of the object. And because we have lidar allover the car, it shows whats happening around the vehicle in realtime. Host is this car communicating with anybody but itself . Raj its capable of communicating with classic cars and other vehicles with similar roos. Its a automated vehicle. T. A. V. In short. Host i also see some cameras inside this car up here. What are these . Raj we had six laser sensors. Radars eras and three as well. One is inside the cabin. One is pointed downwards looking for landmarkers on the road. The other looks at traffic lights. The third camera at the back of the vehicle is for backing purposes, you can see whats going on. And there are also six radars. Theres one behind this cadillac emblem. It is the metal we replaced with the metal logo with a plastic logo so the radar can see behind the plastic. Theres one also thats behind the bumper. The bumper is made of plastic so the radar can see through the plastic as well. Theres radar on the other side of the bumper and there are two on the side but you cannot see from the outside. Internally theyre enclosed. You dont see in the inside either. Theres radar at the back in a plastic bumper also. Host is this car seeing 360 degrees . Raj it is seeing 34 3 0 degrees all the time, as , who have to ns turn their heads around. Host when you get in this car, whats different in the look than a different cadillac . Raj we tried to make this car look normal on the inside and outside. On the inside it pretty much operates and looks like a normal car. Just like you would rent a car at the airport and pick up the keys and the layout looks slightly different but were still able to drive. Its basically the same thing. Ou bring in your keys, get the vehicle running and start driving. Looking at the dark board there are two things that have changed. A button on the dashboard, an emergency stop and theres a button behind the stick shift. Think of this as the autonomy equivalent of Cruise Control engage button. So you would actually engage this button to go into autonomous driving mold. What you need to do is rotate this switch then pull it upwards to get into autonomous driving mode and that is a very conscious action on the drivers part. Host is this car licensed to drive the streets of pennsylvania . Raj in pennsylvania, the laws allow a vehicle to drive itself as long as two conditions are satisfied. Number one, a licensed human driver in the drivers seat and number two, that human can take over control at any point in time. On those two conditions, vehicles can drive themselves on pennsylvania public roads. Host but this could drive as a normal car too, correct . Raj of course. This autonomous board, you push this down. But you would not have the time do that, you can turn it and press the brake pedals or the gas pedal and the vehicle would still respond. Something could be happening that allows the human to take over. And then this button here is only for strictly emergency purposes because we added a bunch of sensors, computers and motors to the vehicle and ge, for example, totally something totally unexpected happens. But the reek is already driving. Ou push that button in, it mechanically, electronically stops the car at that point in time. Knock on wood, we havent had to engage that when driving yet. Host whats the cost of all the different systems youre added . Raj balls many of the sensors, units, we use are one they tend to be expensive. The real idea to be thinking about is that when the volumes go up, the costs will go down significantly. Our thinking is that when these vehicles are mass produced, it would add about 5,000 extra on top of the cost. Host youre going to give us a driverless drive. Is it truly driverless . Raj it is an automated vehicle. It can drive itself under many conditions but not all conditions yet. Not yet. So lets start driving. Ill explain a few more things. Host and youre driving manually. Raj im driving manually. It says manual on the screen. Ill point out a few things on he screen. Lls us that the screen interface. So i guess if i flip back and forth. This is a screen that we added. By flipping a switch you can go back and forth. Host what are these images were seeing here . Ramona this is the display which shows what the vehicle is doing at any point in time so we as humans can be comfortable that the vehicle is indeed doing the right thing. O what we see here is some i conls here and the icons here and the icons for example say i can launch, stop. Tell the vehicle to go to the airport, go home or go to work. And this basically lets you zoom in or zoom out. And what you see on the screen lines. See two blue they are the lane the vehicle can drive in host there are no lines on this parking lot were in. Raj we have a map of the parking lot that allows to us drive along these lanes and we go to the main public reads. That represents the map that your g. P. S. Has. Then you see a green line there. All the people who basically ride around this car and thats the route that your g. P. S. Device calculates. Map p. S. Has a bitin database. You also see a very short red line up there. That is basically the car knowing the green line route, knowing the blue line maps. T basically uses its sensory data from the radar cameras and basically says for the next 15,20 meters, this is how, where, im going to drive. Host dr. Raj rajkumar did you have to Program Every route ahead of time . Raj it needs to have a map of the roads. Host a g. P. S. Map. Would that count . Raj you can think of this as a g. P. S. Map, g. P. S. Navigation device and then you tell the system where it you want to go. It uses your navigation and your calculation for the route. And the red line is what the vehicle is deciding in real time. The red line. Now youll see if i zoom back a little bit. You see a bunch of dots on the screen. Those dots are the laser point from the laser sensors updating in real time. So what you see here is basically a bunch of yellow dots. Host is that that white car . Raj thats the white car. And this dot dumpster there. And you basically see a host and all these white lines are the trees. Raj basically its able to sense in this case the rampedse and cameras and lasers act as the eyes and ears of the vehicle. Host how far can it see . Raj 775 meters. It has builtin Wireless Communications roos that can go as far as 600 meters. Host all right. Raj so lets engage the vehicle in autonomous mode. The vehicle is in parking mold. Im going to engage autonomous mode. Host while its in park . Raj while its in park. Autonomous driving. Ost it started driving. Does it ever make you nervous . Raj i guess the normal reaction for anybody new is at this is that you feel sedate anxiety and angst about not being in control. Its a very normal reaction but lets see host ok. Dallas, you doing ok back there . Dallas i think so. Host its turn signals. Because you told it where you wanted to go already, correct . Raj yes. Host all right. Coming in host so you hit the brakes . Raj i did not. Even though the legal speed limit is 25, pretty much everybody drives at 35 or 40. But this is a stickler to the rules. So basically right now we have a vehicle behind us and of course the driver doesnt like driving so slow. Host uhhuh. It seems to do a little meandering in the lane. Is that a Fair Assessment . Raj it could be better, yes. Host but hes reading constantly. Raj yes. This is a winding road. Host uhhuh. Raj so im not controlling the steering, brake pedal or the gas pedal and it was able to shift transmission by itself. Jorge i see that. A biker . Raj yes. Host all right. How far have you come in 30 years . Raj we have come a very long way but still some ways to go sically completely host is this vehicle constantly learn something raj this vehicle is not constantly learning but its collecting data and using the data to teach the software about new features and functions. Its not learning actually driving. Its learning after the fact. Host how did it know theres a stop sign there . Raj the map basically has indications about where the stop line is. Host that was the car that did that . Raj the car did it all by itself, yes. Host it wasnt sure of its speed . Raj basically saw the parked cars. There are two ways to get back. Ill take over manually. Ill just push this down. The vehicle has gone back to manual mold. Autonomous ready. Raj it says autonomous ready. Jorge and can you do that on the fly . Raj yes. You can switch back and forth seam lesley. Host and its seeing all of these things . Raj yes. This crosswalk is not on the map. Host oh, ok. Raj these lanes have been changed recently. The maps are have not been completed. Host ok. So its not quite ready to be sent out on a road its never been on before. Raj we have done that on highways. Highways weve never been on before. In the city, bicyclinging and more things. We do that on the highway, not in open corridors. Host ok. A bike. Raj ill zoom and you can see it better. We are back on the curvy, winding road. Host kit read signs . Raj it can read some signs, yes, but not all signs. There are thousands of signs. It does not understand all of hem. So now we see that red light well. The green is the stop line. The red line is exactly on top of that. Host how far have you driven in this car autonomously . Raj a total of about 20,000 iles awe awe tom mousely. Host whats the longest stretch youve ever taken . Raj weve done a couple hundred miles of highways. A system thats been used to basically drive from San Francisco to new york city, about a 3,500mile journey and the vehicle drove itself on highways about 98. 6 of the time. Used ology had been so highways are not a problem. Autonomous ready. Raj so you have taken your first ride in an Autonomous Car. Host when will we do this regularly as consumers . Raj simple question, basic question and ill give you a long answer. You can already buy vehicles, for example, tesla with an auto pilot feature. The vehicle can drive itself but the human must be paying attention. General motors next year will introduce a similar feature they call super cruise where the vehicle can steer itself and apply the brakes and that will be in a cadillac sometime next year. And many highend vehicles can already drive themselves today. Some of these features are already available on the market and then three to five years from now, we expect that the vehicles will be able to drive themselves but in wellspecified, welldefined geographically constrained regions. Where ly, for example, bicyclists are not allowed and there is no heavy rain or heavy snow. So some of these technologies were deployed earlier but when you ask when can the human not drive at all . That basically implies that the technology should be able to drive the vehicle itself from any point a to any point b that any experienced drivers can drive in the u. S. That capability is going to take at least weve come long way over the past couple of decades or so but still a quays to go before the human can go to the back seat and take a nap. Host have you allowed your kids and your wife to ride with you in the Autonomous Car . Raj sure, weve allowed family members to go along on in of the projects, yes. Host i was a little surprised that we didnt have to sign a lease before we got in. Raj i guess if you were with a corporation, but just because we are [indiscernible] raj host why are we talking to you about Autonomous Cars in pittsburgh rather than detroit or Silicon Valley . Raj thats a great question. Carnegie mellon is globally well known, as a has a strong reputation for computer science, engineering as well as robotics. We have a Robotics Institute on campus. International and has more than 100 researchers in it, excluding students, if you will, and theyre all extremely knowledgeable about the robotics and in the field have been built at c. M. U. Since the early 1980s. In fact, we at Carnegie Mellon believe that we are the birthplace of Autonomous Technology dating back to about 1983 or so a couple of years back in 2014, we literally celebrated the 309 birthday of this technology on campus. Host you said before we starred this interview that computers are simultaneously very intelligent and very stupid. Raj yes, exumpletse are simultaneously very intelligent. They can do things that amaze us, right . They can react very quickly and they can make decision that is finds extremely smart. How does it know theyre driving at this speed and so on. They are really intelligent because they process a 360degree view of the vehicle with the multiple sensory data streams from radars and cameras. Very intelligent but at the same time they are stupid, if you will, because they dont really have common sense. For example, we know that when we fall down or when we basically touch fire it hurts and the next time you wont do it but computers cannot make that dink. Hey, i crashed into somebody last time around, next time dont do that. It will do the exact same thing unless this program through something else, specifically by a human being. Host whats the difference between the vehicles here in the lab. The jeep and the cadillac . Raj we see two vehicles. One is a red jeep. The other is the cadillac that we were able to demonstrate the vehicle driving itself. The vehicle on the left, the ed jeep, is called nav lab 11, meaning this was created by a Research Laboratory at c. M. U. Called nav lab, short for Navigation Lab and the 1212 indicates this is the 11th generation of autonomous ehicle the lab has bill bit built. So there were 11 variations of this vehicle. The vehicle on the right is the cadillac that we were able to drive today. That vehicle has been created by the project that i lead with support from General Motors through the u. S. Department of transportation as well as the National Science foundation. So because of our close working relationship with g. M. , we are extremely sensitive to the a sthet aesthetics to the vehicle, the interior and the exterior as well. So that it looks very normal, that something g. M. Would be proud to both design, manufacture and sell. Host how far along are we with in technology . Raj the technology has come very rapidly in the past veral years since the 2007 car. The 2007 challenge basically the outcome was that 16 competed and our team from Carnegie Mellon won the competition. It demonstrated once and for all that the notion of a vehicle that can drive sits no longer science fiction. It demonstrated beyond a shadow of doubt its only a question of when not. They will then, hire key people from Carnegie Mellon, key people from Stanford University and thats how google launched its division. Google basically has publicized when hnology and then Car Companies like uber came into being, they have a Financial Economic incentive to have the cars drive themselves and meanwhile the carmakers, General Motors and ford in the u. S. Audi, b. M. W. In europe. Nissan, honda and toyota in japan. A whole bunch of chinese car makers who want to have a part in the market and one in south korea, all of them have been investing big time in this space. Bosh, a company in germany, as well as delphi, based in the u. S. , have been investing as well. We have investments, research and portfolios. Nd from technology from uber and so on so technology is progressing very rapidly. Host sit competition . Are they developing their own systems like you are here . Raj thanks to capitalistic forces, that is a there is a lot of competition. This market senior selfdriving vehicles is expected to be huge in 15, 20 years or so all these companies we just discussed or pieces of the technology if not become leaders. Thats what the competition is all about. Host what about the city of pittsburgh . Has it been supportive . Raj the city of pittsburgh has been a topnotch supporter of this technology. Weve been driving a selfdriving cadillac on the Public Schools of public roads of pittsburgh since 2007. The state of pennsylvania has been friendly to the technology as well. Pennsylvania allows us to drive with this technology as long as theres a human licensed driver in the drivers seat and the drive can take over at any time. Besides, that, the vehicle can drive itself and because theres a human in the drivers side, the human be would be liable. Host whats surprised to you since 2011 when you built this cadillac . Raj that we were able to build a vehicle that for the most part looks normal on the inside and outside but we were still equipment all the so the technology can be made an Additional Technology cal caliber. Host what have you taught the computers or the entire system since 2011 . Raj the system, the vehicle to drive itself needs to have sensors like cameras, radars and lasers. And basically needs to have motors of some kinds that steel the uheel and can emlate braking and acceleration actions. All the Data Collected by the sensors have been processed by a bank of computers but all the Data Processing happens by software so that literally hundreds of thousands of lines f software that goes through these data streams sends a property of commands to the steering well and wheel and to the pedals. Host and all the raj so all the magic at the end of the day is in the software a. I. Or artificial intelligence. Host 10 years from now are we going to look at this cadillac and sayreville this was a nice back plaque and white tv . Ramona yes, 15 or 20 years from now people will say look at how quaint that vehicle was. Host whats the difference between an automated car and a car . Raj auto maas Autonomous Car is a car capable of driving itself typically using sensors and computers that are puts into the vehicle. It senses whats happening in the environment and then has computers that reacts to the data coming in and then sends local commands for steering and braking and so on. A connected vehicle is something that is capable of communicating with the cloud, with traffic lights that are roperly equipped, with traffic vines as well as other vehicles. Your smarts phones, laptops and desktops can communicate with each other. They can, why can not a vehicle . It can. It can commube indicate with the environment. So a connected car is capable of you know indicating wirelessly to others in the environment. Telling where it is, who the, what it plans to do and sayer y wire information other information can be received wirelessly so it can generate a warning to the human driver so if the human season not able to see, for example, a strong vehicle like in the dark. The wireless communication will let the human know that theres a problem up ahead, slow down. That the roads are slick, slow down. Studies incade indicate that p to 80 of accidents can be prevented by using this technology. It can be generating warnings and aletters to whats human but its possible you can actually combine the two. A connected Autonomous Vehicle so there you get the benefits of both, connected and automation. Host window that take a big investment in u. S. Infrastructure, to get connected cars on the road . Raj great question. It basically requires a small, wii ilike device fivelike device that sends messages. They basically cost about 100 each. Very, very inexpensive and it can go out as far as 600 meters so for 100, you see much farther and you can literally see as well because a radio wireless base can bounce on buildings and come to you as well so it gives you this super human mission, if i will if you will. In the u. S. , each state has its own jurisdiction. They can come up with their own local law that is operate within the state. You get a license from the state that you can drive. When you have a license from pennsylvania, you can also use the same license for driving in ohio and california. The states at the end of the day have the final say. If each state comes up with its own rules and regulations for automated vehicles, it would be a complete nightmare for car manufacturers. Once you cross the state border, you lose supplies. The national Highway Administration is looking to provide a set of guidance, if you will, to all the states so they can be harmonized so if i can operate in one state, hopefully the same technology can be used as it is in the neighboring states. Host professor raj rajkumar, youve put maybe 20,000 miles on the cadillac driver lesley . Have you gotten to the point where youll look away from windshield wile its driving. Yes, when we accumulateed that 20,000 miles. It so happens when we do demonstrations, sometimes public, sometimes v. It. And with a camera watching me off the time and then i take my eyes off the road or any hands off the wheel, they get it and highlight it. So that happens subconsciously, yes. Host whats the main reaction you get from people riding in the car with you . Raj most if not all people who are riding in a driverless car for the first time, the typical reaction is one of anxiety and angst. Here and occasionally panic attacks but then they basically watch that the vehicle is able to drive, exactly stopping when it should stop. Its actually taking the cold fairly comfort my and they build a agree of confident. They are watching the vehicle very intently. What is it doing . After a while they say a fewer more minutes go by, i start conners ising with conversing with e. P. A. Other people in the car and after a while i stop noticing what the car is doing. This takes anywhere between five and 15 minutes. So the concern that researchers like me have is not that people are fearful. The concern that i have is that people become too comfortable no, quick quickly. So the human still needs to be alert and pay attention. Host it seems all of a sudden all of this technology is kind of whooshing towards us. Is that just a perception because of the media . Raj because of the huge market potential that exists for selfdriving vehicles, Many Companies or many destroys are interested. Number two, thanks to advances made, people know that the technology is viable. Everyone want to wash wants to basically benefit economically from that so that is a significant amount of investment on the Technology Side and theres basically a lot of aggressive marking so to bet the word out that each company is on a particular piece of technology and stuff. So the potential for economic grains gains in the how much and the market is basically what youre seeing today. Host what about sexurts privacy . Raj because there are Different Computers and hundreds and thousands of ftware corps we have to be cognizant of and be cautious of possibly of the trolling. These could include cybersecurity attacks, where i could send informationout and i could also receive back in. Those points become potential folkals for attack. So we need to be careful. Whether the attack comes from across the street, the continue, the country or globally. One also has to be worried about what can be done in the physical context. For example, you can jam g. P. S. , you can possibly spoof gmple perform s. Lasik device you can use very simplery so that cybersecurity and that also physical attacks that are possible so we elected to combine the two and basically said cyber typical security problems. Host finally, doc, whats the jex general next generation . Are you working on the next generation of technology for this sneak yes, i think of the first generation as being 2007, then it was conducted. They all had very similar sensors, if you will. Light out on the roof of the carring being an example. The cadillac you see behind me. Think of as salvation two. The next genres is smog were focusing on. Use automobiles to create technologies completely. They will be able to deal with a lot more scenarios on roads and they have got to be table to drive on roads that theyve never seen before. Host whats your biggest frustration with this technology . Raj i guess theres been its been very safetying so its not that i have many frustrations. I dont call it a frustration but its the challenge of dealing with the inhernt silliness of a computer. What does sexoveb seedy pick aly immediate to be talk. Painstakingly more computers so that it just takes time. Host raj rajkumar of Carnegie Mellon. Thank you for your time. Raj its been a pleasure to talk to you, peter. Next week, the communicators looks at connected connected cars at a Research Location in michigan. If youd like to see some of our previous the communicators programs, go to cspan. Org. In 1992, President TelevisionCable Companies and brought to you as a Public Service but by your satellite or cable provider. The second president ial debate is sunday night at washington gluferte st. Louis, missouri. Watch our live coverage for a preview of the debate and then at 8 30, the predebate viewing in front of the office. Then the live debate followed by comments. Atch live or on demand using our desk phone, or tablet. House speaker powell ryan joined senator ron johnson and Governor Scott walker earlier today for a republican fundraiser in wisconsin. Donald trump was originally set to attend the event but cael