Next on the communicators a conversation on selfdriving cars. The candidates were back on the campaign trail following last nights president ial debate. This week the communicators goes to pittsburgh to talk about selfdriving cars. We take a ride in the Autonomous Car that the university is developing. Host we will talk to the communicators of Carnegie Mellon. What is this . Guest we are looking at a cadillac suv from 2011 which is capable of driving itself so its a selfdriving cadillac. Host if you look at the card looks pretty normal from here. Guest that is the car where it that we literally bought new from the cadillac dealership and bo did a bunch of censors and computers and it drives itself. Host do you want to give us a ride . Guest i would be happy to give you a ride. Hop in. Host what is the main reaction you get from people writing in the car with you . Guest the typical reaction is one of anxiety, angst, fear and occasionally even panic attacks. But then they basically watch the vehicle that is able to drive, stopping when it should stop and its actually taking the curve comfortably. They build a degree of comfort. We learn more about the move to selfdriving cars and professor ragnunathan rajkumar at the lab where were his experimental cars been worked on. Host . Or the communicators what is your job at Carnegie Mellon university . Im a professor of engineering and robotics at Carnegie Mellon university. Host how did you get into that . Guest i did my post graduation studies and my masters in ph. D. At Carnegie Mellon and then i joined ibm research and i came back because i like it that much. Host what kinds of things do you work on . Guest i work on what are called systems. These are things that embeds computers that but use a different device. For example your phone is basically a device with computers inside of them but the body of the computers inside him. Let me show you an example. Its meant to be a television. Think about a project or. Protector has computers inside of them but you think of it as basically that device. That we use on a daily basis that causes cost 10 bucks as an embedded computer embedded inside of the toaster. It embeds computers but its a dedicated device of some other kind. Ive been working on embedded systems since my doctoral and it turns out intelligent vehicles themselves are classical example of these systems so a car is a car that transports people and goods from point a 2. B. It just does that with computers embedded inside of them. Host how did you get into the business of Autonomous Vehicles . Guest a great question. I had been working with General Motors, the carmaker since 2004. I have been working with researchers in our Department Since year 2000 so i started working with the gm group and started applying my embedded systems into automobiles and in 2006 the Defense Advance Research agency and the u. S. Military announced a competition called the urban challenge. Vehicles that drive themselves. They need to drive for 60 miles in fewer than six hours driving in an urban like setting interacting with other selfdriving makers as well as human and following the same rules of a road that you and i have to follow on a daily basis. But that competition, gm became our biggest partner and sponsor. We had about 23 other sponsors as well but gm was the biggest of them. We had have a very strong working relationship with gm and became an integral part of the team that worked on a vehicle which ended up winning the competition as well. Thats how i got to be a documented maker and when our theme from Carnegie Mellon won the competition gm which was the biggest sponsor the team said hey our Team Sponsored the winning team and they said because its about urban driving it clearly has implications to the consumer Vehicle Market segment and we started a second lab on campus focusing exclusively on automated driving and ive been running that lab as well since it was launched so thats how i got to work with gm. My relationship with gm became extremely strong and very loyal. Host so does gm on the technology that you developed . Guest the Tech Knowledge he that the sponsor, their support is owned by Carnegie Mellon university and we have Licensing Agreements with gm who have access to parts of the technology. We learned more about the experimental cadillac as we got ready to take it for a drive. Host i want to start by looking at this. What is this monitor thats up on the top . Guest that is one of six laser sensors called lidars there is one on before head of the car and there is one on the bumper here right there. Thats a second lidar in the third lidar on the other side of the bumper as well is in the back window of the car and theres one on the other side exactly opposite of that. Host what id do a reading . Guest what they are doing is sending out multiple laser beams and when the laser beams hit an object they bounce back and come back to the transmitter and because we know the speed of light we can actually calculate how far and because of the multiple teams that create scanning as well you get a profile of the obstacle that the vehicle just encountered and because we have lidar all over the car the lidar is telling you what happens or a vehicle. Host dr. Ragnunathan rajkumar and this is car commuting communicating with anybody anybody but self . Guest is capable of actually communicating with a properly equipped traffic light, traffic sign and other objects with similar ready or was radios. Host i also see some cameras inside the car right appeared. What are these . Guest in addition to the six lasers sensors we talked about there are three cameras in six radars as well so they are two cameras one on either side of the rear view mirror. One is pointed downward looking for land markers on the roads and other cameras are pointed upwards looking at traffic lights. The third camera on the back is tracking focusing see kinsey was going on. There are six radars. There is one behind this cadillac emblem. This is a medical local with a plastic logo so you can see through plastic but it turns out the radar behind this bumper as well on top of the lidar behind the bumper, the bumper is made of plastic so the radar can see through the plastic as well. There is radar on the other side of the bumper and to on this side right next to the lidar you dont see it from the outside and he dont see from inside either. Theres another radar on the back. Cisco is this parsing 360 degrees . Guest existing 360 degrees all the time as opposed to humans where our heads would have to turn all the way around 360. Host professor raj rajkumar what is different in the luck than a regular cadillac . Guest we try to make this car look normal on the outside and the inside. On the inside i pretty much it pretty much operates and looks like a normal car. Just like when you pick up a car at the airport but its able to drive so its basically the same thing. You start the car manually and start driving. Meanwhile if you look at the dashboard to things have changed. That is a button on the dashboard. Its an emergency stop and a button by the stick shift. Think of this as the autonomy equal of Cruise Control so you would think age this button to go to autonomous driving mode. The engine rotates the switch and pulls it upwards to get into an autonomous driving mode and thats a very conscious action on the drivers part. Host is this car license to drive the streets of pennsylvania . Guest in pennsylvania the law allows a the vehicle to drive itself as long as number one a licensee among drivers in the driver seat and member two that acumen can take over control at any one time so under those two conditions they can drive themselves autonomously. Host this good drive of the normal car as well . Guest this goes from an Autonomous Vehicle but if one did not have the time to do that you could turn it press the brake pedals in the gas pedal and the vehicle still response. Any number of things could be happening for humans to take over. This button here is for emergency purposes. We added a bunch of sensors and computers and engaged for example something totally unexpected happens like you start smelling smoke and you have no idea whats going on. The vehicle is driving come he does push the button and ended electronically and mechanically becomes knock on wood. Host what is the cost of all the different system you have added to this car . Guest many of these sensors, radar and the cameras are one unit. They tend to be expensive so its an expensive vehicle because of that but its an idea to be thinking about. When the volume goes up the cost will go down significantly. Our thinking is that when the vehicles are mass produced it would add 5000 on top of the price. Host you are going to give us a driverless ride. Is it truly driverless . Is that a fair word to use . Guest this is an automated vehicle. He it can drive itself under many conditions but not under all conditions, at least not yet reflect start driving and i will explain a few more things. Host and you are driving manually . Guest i am driving manually. Ill point out a few things on the screen. If i switch back and forth and stopped the vehicle, this is the screen and by flipping a switch you can go back and forth. Host what are these images we are seeing here . Guest what you will see on the screen, so this is the display that shows what the vehicle is doing at any one time so we as humans can be comfortable that the vehicle is indeed doing the right thing. What we see here are some icons here and the icons for example say that i can stop, i can make the vehicle go to the airport or go home and it basically lets you zoom in or zoom out. And what you see on the rest of the screen is you see two blue lines which are the lanes you are driving. Host there are no lanes in this part do not. Guest you go to the main main and that sparks the gps device. The blue line is the map if you will and you see a green line there. We have a program that shows where we are and the gps device calculates it. The gpas isnt a map database. That is the green line that you see. You also see a sharp red line right there. That is basically the car knowing the green line route, knowing the blue line map, it basically uses that sensory data from the lidar camera and over the next 15 or 20 meters this is where im going to drive host dr. Rajkumar do you have to Program Every route ahead of time or can it go to a street that has never been on before . Guest it needs to have a map of the roads. Host a gps map. Guest a gps map and then you need to basically tell the system where you want to go. It uses the map information and the Current Location to calculate the route using the gps device and google maps. The red line is deciding so i guess now you will see that i will zoom back a little bit. You see a bunch of dots on the screen. Those dots are the laser points or the laser sensors so what you see here is basically a bunch of little dots. Host is that a white car . Guest thats a weichert and these blue colors respond. Host and all these white lights. Guest basically these are her eyes and her ears. The camera and radar acts as the eyes and ears. Host how far can it see . Guest you can see 175 meters but the technology we discussed earlier the wireless communications. Your radios doppler and so on can go as far as 600 meters. Host already. Guest lets engage the vehicle in autonomous mode. The vehicles in parking mode. Im going to engage in autonomous mode. Host while its impart . Guest while it is in part. It has started driving. Host does this ever make you nervous . Guest you feel a lot of anxiety and angst. Its a very normal reaction. But lets see how it does. Host okay. Dallas, are you doing okay back there . I think so. Host at turns on the turn signals. As you told the where you wanted it to go already, correct . Guest yes. Host all right. So you hit the brakes. Host no, i did not. It hit the brakes. Host and it does the speed limit. Guest yes, its exactly 25. Pretty much everybody drives at 35. Basically right now we have people behind us and of course they will say why are you driving so slow . Musculus seems to be doing a little meandering in the lane. Is that a Fair Assessment . Guest it could be better, yes. Host but its reading constantly. Guest yes. This is a curving winding road. So im not controlling the steering well at the break pedal are the gas pedal. It is shifting the transmission by itself. Host i see that. Its sensed that biker . Guest yes. Host all right. How far he had become in 30 years . Guest we have, very long way but there is still some ways to go to remove the human from the driving. Host is this vehicle constantly learning . Guest this vehicle is not learning. You can expat and it uses the data to feed the software. It is not learning as we drive. Learning afterthefact. Host how did another was a stop sign there . Guest the map basically has locations on where the stop signs are. It did that all by itself. Host that was the car that did that. It wasnt sherbet speed . Guest up when slightly to the right. I will take over manually. Ive pushed this and in this case the vehicle has gone back into manual driving. Host can you do that on the fly . Guest yes. Host its seeing all of these things. Guest these lanes have changed recently and its not on the maps. Host its not quite ready to be sent on the road is never been on before. Guest highways, highways that its has never been on before can do that. But roads with bicyclists and strollers,. [inaudible] host a bike. Guest we are back on the curvy winding road. Host can it. Signs . Guest it can. Some signs, yes but not all signs. It does not understand all of them. So whenever you see that red line, you see the green line and is the red line is exactly on top of that. Host how long have you driven this car autonomously . What is the longest trip you have ever taken . Guest we have done a couple hundred miles on the highways. The vehicle is used by us enough that i created, basically it drives a 3500mile journey and his highways about 90. 6 of the time. The highway is not a problem. So you have taken your first ride in the car. Host when will we do this regularly as consumers . Guest simple question, basic question and i will give you long complicated answer. We can our day by vehicles for example tesla with an autopilot feature that can drive by itself but the human must be General Motors next year will create a similar feature that they call super cruise that can steer itself and apply brakes as well and that will be the cadillac cts sometime next year and many highend drive themselves today. Some of these features are regarding available and three to five years from now we expect the vehicles will be able to drive themselves but in well specified welldefined geographically constrained areas and its called geofencing. For example bicyclists are not allowed and there is no heavy rain or heavy snow. They could deploy these technologies earlier but when can the human not drive at all implies that technology should be able to drive the vehicle itself back any point in maclennon point b in the u. S. That capability will it take at least 10 years. We have come a long way over the past couple of decades or so but theres still quite a ways to go before the human can take himself or herself out of the drivers seat go in the backseat and take a nap. Host have you allowed your kids or your wife to ride with you in the Autonomous Car . Guest family members and many other people go in the car, yes. Host i was a little surprise that we didnt have to sign a release before we got in. Guest thats because we are researchers. Host why are we talking to about Autonomous Cars in pittsburgh rather than detroit or Silicon Valley . Guest Carnegie Mellon is a globally wellknown and has a strong reputation for computer science. Engineering as well as robotics. There is a Robotics Institute on campus and it has more than 100 researchers in it excluding students if you will and they are all extremely knowledgeable about the area of robotics and intelligence in the field. In fact we have Carnegie Mellon believe that we are the place of a automatic technology. In 2014 we literally celebrated the 30th birthday of this technology on campus. Host you said before we started this interview that computers are simultaneously very intelligent and very. Guest yes. Computers are simultaneously very intelligent, they can do things that amaze us. They can react very quickly and they can make decisions that the normal person find extremely smart. How does it know that it should be driving at the speed . They look very intelligent and they are very intelligent because they process the view of the vehicle with the multiple sensory data streams from lasers and cameras. Very intelligent. At the same time they are if you will because they dont really have common sense. For example we know that when we fall down or when we basically catch fire at hurts and we wont do it. Computers cannot make a simple inference. If you crash, next time dont do that. Unless its program specifically by human being. Host what are the differences between the two vehicles that we see in the lap of a jeep that seems to have a lot of equipment on it. Guest we see two vehicles. One is the red jeep and one is the cadillac. The vehicle on the left, the red jeep was created by a research Love Laboratory and the 11th generation of autonomy. So they are 11 versions of the vehicle so that he had loaned the right is the cadillac that we were able to drive today. Thats vehicle was in the project that i lead from General Motors as well as the National Science foundation. Because of our close working relationship with gm we are extremely sensitive to the aesthetics of the vehicle, the exterior aesthetics in the interior athletics as well. It looks very normal and is something that gm would be proud to design manufacture and sell. Host how far along are we with this technology . Guest the technology has come very rapidly over the past several years. It has come very far. The 2007 challenge basic we the outcome, our team from Carnegie Mellon won the competition. The notion of a vehicle that can drive itself is no longer science fiction. It has demonstrated beyond a shadow of a doubt that its a question of when, not if. Since then google hired key people from Carnegie Mellon, key people from San Bernardino city and thats how google launched itself driving vehicle project and then google basically enlisted several hundred and technology and Car Sharing Companies like uber. We have a Financial Economic incentive to basic layout they have the vehicles drive themselves and meanwhile the carmakers, General Motors, ford and the u. S. , audi, bmw and daimlerchrysler in europe, nissan honda and toyota in japan, a whole bunch of chinese carmakers who basically want to have an edge in the market and south korea all of them have been investing bigtime into this space and meanwhile big automotive suppliers like continental and germany as well as the u. S. Have been investing bigtime as well so we have a lot of action in terms of investment, research and creating Technology Portfolio some karmic is, the supply chain and technology Companies Like google, opera and apple. Technology is progressing very rapidly. Host you talk about it is action that are they developing their own systems just like you are here to ask. Thanks to capitalistic forces there is a lot competition. Its expected to be huge in 15 or 20 years or so so all these companies that we just discussed are pushing big pieces of the technology. Its about establishing it. Host what about the city of expert . Has it been supported . Guest the city of pittsburgh has been a topnotch support of technology. We have been driving on public roads in pittsburgh since 2011. The state of pennsylvania has been using the technology as well. They allow us to basically drive vehicles with this technology as long as theres a human licensed driver in the drivers seat and a person can take over the control at any point in time. Besides that the vehicle can drive itself and if something goes wrong because theres a human and drivers seat the human would be liable. Host . Or rajkumar would it surprise you since 2011 when he bought the cadillac . Guest so what has been satisfying if you will is that the vehicle for the most part looks normal on the outside and the technology for sensing and computing and steering the will and pressing the pedals. Itec elegy can be made aesthetically pleasing as well. Host what if you talk to computers about the entire system since 2011 . What Additional Information has gone into it . Guest the system, for the vehicle to drive itself that needs to have sensors like cameras, lasers and radar and these basically have models of some kind of steer the wheel and can basically break but at the end of the day all the Data Collected by the censors have been processed by a bank of computers. All the Data Processing happens by software so literally hundreds of thousands of lines of software and data streams apply to the rules of the road and then sends proper commands to the Steering Wheel into the pedals. All the intelligence if you will inc. Into the software so at the end of the day its the software and artificial intelligence. Host 10 years from now are we going to look at this cadillac and go that was a nice black and white tv . Guest guests, 10 or 15 years enough people will talk about Lookout Point that little vehicle was. Host what is the difference between an Autonomous Car and they connected car . We were in m city in ann arbor recently as well. Guest an Autonomous Cars or car car that is capable of driving itself typically using sensors and computers that are built into the vehicle so it depends on the vehicle itself. It depends on whats happening in the environment and it has data coming in from the censors and local commands for steering and braking and so on. Thats an Autonomous Vehicle. It connected vehicle is something that is capable of communicating with the cloud on the internet with traffic lights that are appropriately equipped with traffic signs as well as other vehicles. Your smartphone, your desktop can communicate wirelessly with each other. If they can why cant a Vehicle Computers to talk to the internet . It can talk to the environment so connected car is capable of communicating wirelessly with others in the environment on what it plans to do and similarly the empire meant sends the information wirelessly sync the traffic light just turned red and it can basically say it will start the vehicle and the lane ahead and all the information can be received wirelessly. It can generate to the human driver. The wireless communities and slept the human know if theres a problem to slow down. The roads are slick ahead, slow down. Its a huge safety addition if you will. Studies indicate that up to 80 of accidents can be prevented by using connected technology. Connected technology can generate warnings and alerts so its possible they can combine the two. It is connectivity and nomination. Host would that take a big investment in u. S. Infrastructure to get connected cars on the road . Guest a great question. Connectivity requires a small wifi like device that we all use on a regular basis. One of those devices actually costs only about 100 each. Its very very inexpensive. Anything that the laser the radar and camera can see so for 100. A radio wireless base can come to you as well so it comes to the superhuman mission if you will. The capability command and low prices compared to the rest of the technology. They can come up with their own local laws that operate within the state. You get a license from the state that they drive and you have a license from pennsylvania you can use the same license in ohio and california. It would be a complete nightmare for car manufacturers. So the national Highway Safety administration is providing a set of guidance if you will so they can be homogenized or harmonized so if i can operate in one state it will be this same tech elegy in a neighboring state. Host professor rajkumar up maybe 20,000 miles on the cadillac, autonomously. He gotten to the point where you will look away from the windshield while its driving . Guest rehab simulated 20,000 miles. We do demonstrations so we have sometimes public can sometimes we have people in the car and cameramen watching of the time. I take my eyes off of the road and the ability to get an in highlight. That happens subconsciously, yes. Host what is the main reaction you get from people writing the car from you . Guest its a remarkable sequence that happens. Most if not all people after writing in a driverless car had written for the very first time. A typical reaction is one of anxiety, angst, fear and occasionally panic attacks. But then they basically watch the vehicle is able to drive stopping when it should stop. Actually takes the curve fairly comfortably and recognizes and builds a degree of comfort. We are watching the vehicle very intently. What is it doing . After a while the dealer says its doing what it needs to be doing. A few years go by and they start conversing with other people in the car and after a while they start noticing what the car is doing. This acute anxiety to insurance up at the vehicle is doing takes five to 15 minutes depending on the person so the concern that we have is not that people are fearful. People become too comfortable, too quickly. So the Steering Technology has quite a ways to go in a situation that they encounter in a daytoday basis of the human still needs to be. Host seems all of a sudden all of this technology is pushing towards us. Is that just a perception because of the media . Guest because of the huge market potential that exists Many Companies are interested. Number two, the advances made are challenged. People understand that the technology is viable and they benefit economically from that so thats a significant investment on the technology side. Basically a lot of aggressive marketing if you will to get the word out that each company is working on technology so the combination of the advances, the potential for economic gains in the future and the Marketing Initiative is basically what youre seeing today. Host what about security and privacy because so much of this technology is out there on the airways. Guest because these are driven by computers and hundreds of thousands of lines of software we have to be cognizant of and be cautious about threats to technology. These could include cybersecurity attacks. I can send information out. I can also receive information back in. Those entry and exit points become potential portals for attack. We need to be careful that the attacks coming from across the state, the country the continent or globally. The cybersecurity threat one also has to be worried about what what can be done the physical context. For example it can jam a gps and possibly spoof the gps and use for example even a simple laser device you can buy cheaply to confuse the lidar censors so the cybersecurity threats and physical attacks that are possible so we like to combine the two. Host finally, what is the nextgeneration . Are you working on the next generation of technology for this vehicle right now . Guest we are focusing on the capabilities. Nextgeneratis being the conductor and if you look at the they all had very similar censors if you will a streaming lidar on the roof of the car being example. The cadillac you see behind me think ofhe