Driven - Mobility Moment: A Discussion on Infrastructure for Autonomous at AutoSens
Hello and welcome to Driven's Mobility Moments podcast, where we talk with the people building the mobility ecosystem in the Detroit region. Today we have a special podcast for you, a panel discussion from the AutoSens conference which took place this spring in Detroit. I'm talking with Shawn Brovold
, market development manager with 3M Connected Roads, Transportation Safety Division. Abhay Rai
, vice president of product marketing with Visteon Corporation, and Ana Dalsnes Storsaeter
, senior engineer and project leader with the Norwegian Public Roads Administration.
We are talking infrastructure and future autonomous vehicles. What do we need now? What will we need in 20 years? How do we create standards? And, who is missing from the connected infrastructure conversation? I'm your host Claire Charlton.
So here we are, we're in the Earth Room at the Michigan Science Center, and we're at a conference called AutoSens, which is an annual conference here in Detroit, but also in Europe and Asia--soon to be Asia. And it focuses on all things related to sensors for ADAS, connected vehicles, autonomous vehicles, so that can be radar, it can be Lidar, it can be cameras, it can be other fun things.
I have some fantastic guests here today, and we're just going to go around the table and introduce ourselves, and then we're going to talk about some cool infrastructure things. Which I think are really important. So Shawn, why don't you go ahead and introduce yourself?
Sure, yeah my name is Shawn Brovold, I kind of have a pretty unique background in that I worked in the automotive industry for quite a while. But I also have a background in civil engineering, and so I've also worked in the transportation industry on the civil engineering side. So it's kind of a unique position that allows me to look at how to approach some of the solutions to overcome and to benefit automated and assisted vehicle technologies both from the infrastructure side, as well as the automotive side.
So really we're trying to bridge the gap between the two. And I work for 3M in a group called the Connected Roads Group. And we're looking at essentially taking some of our existing infrastructure products and futurizing for the next generation of vehicles.
Cool. Really cool. And complicated.
Yeah. I think one of the more complicated parts of it is, how do we develop something that's practical and that's ready for deployment in the near term, and that can provide a benefit for both existing drivers as well as future automated systems?
The future. Abhay?
So, I have an interesting background too. My name is Abhay Rai, I'm a VP of product marketing for Visteon. I joined Visteon around three months ago. And I relocated here from the Bay Area, where I was head of automotive sensor marketing for Sony. I was a design engineer so I have a design engineering background. I was IC design engineer, those silicon design engineer. And they design so many products, including multi touch products that go in the phones. I worked on mouse sensors that still touch a lot of hands.
And then I went to business school and then slowly transitioned into marketing. And it is very interesting, the way you started this discussion, in that how technology and Detroit will intersect because Detroit is traditional automotive. It's not easy, 30,000 parts, very complicated. And then Silicon Valley that thinks that it can change the world with technology. So how you intersect? And for two, three years we saw that all the Lidar companies were saying that we can change the world. Now they realize how hard it is. So I think how technology and automotive experience will intersect.
And that's when I found that Visteon is doing something very interesting. And that's why I joined, that's why I moved. And I think a lot of Detroit-based companies are going in this direction, I hope Visteon can spearhead in this direction because if you look at the value chain, OEM's, Tier 1s and Tier 2s, sensor suppliers and all those are Tier 2s. And if they're not enough value than Tier 1s will get squeezed. So how they are going on the technology map has to do with if you look at the car, 50, 60 years ago, then people were paying based on the weight. How much steel the car has. How much metal it has.
And then when you take the clock forward, people started paying for the car based on the value, based on electronics. Now it is time for software. If you look at the Tesla car, when you go in the car in the evening and you do not have any appointment in the calendar, the moment you sit in the car it drives you to your home. When you have your appointment and 30 minutes before the appointment when you sit in the car, it looks at your appointment and drives you to that distance. So artificial intelligence, it comes from the software. But that software, where the value is, is a software when it is designed without understanding hardware in mind, then basically it's an absolute disaster.
And that hardware, power of hardware is understood very well by Detroit. So I think if we can put this whole thing together, it is a win benefit for everybody.
Wonderful. Thank you. Ana?
Ana Dalsnes Storsaeter
Hi, my name is Ana Dalsnes Storsaeter. I have sort of an opposite background from Shawn, my background is I'm also a civil engineer, in mechanical and IT. And then I moved from mechanical engineering into transportation. And I've been a project leader for projects in the Nordic countries with car-to-car communication, or car-to-infrastructure communication. But recently I've started to research what is needed from our infrastructure to complement these new technologies that we are seeing.
So if you think about building a road, planning it and building it takes many years. And the road is supposed to be there for a few decades. So we always think about 20 years ahead what will the traffic be like? And in 20 years time, I mean, we see some automation now, and that will only increase. And it's natural to think that we already spend a lot of money on roads and infrastructure. So if we can do things to make changes to this, to make it more understandable for machines, then that will be a benefit for everyone.
And I agree with what was said earlier that it's important to see this like maybe in the short-term as well, because many people are thinking about robo-taxis and when cars will drive by themselves. We will have this situation with mixed traffic for some time. And then things have to work both for humans and for machines. And traditionally we are centered around the human, we understand how the human sees everything. And now we have to understand what the machine, how does it work? How does it receive things? Or do things we can do with trail markings or with signs or even geometry that will help the car position itself or understand the environment.
So if I understand correctly, there's going to be a little bit of pain before we get to a more painless experience on our roads in the future.
Ana Dalsnes Storsaeter
Yes, I think this short-term situation with mixed traffic is very complicated. You have a lot of different actors. You have humans who are unpredictable. You have different users, pedestrians, bicyclists, cars, and cars who are driven by humans or machines. And that's the most complicated situation we will have. It will be much easier if we have solely cars driven by machines, or humans. But when we have this mixed situation it's hard because not only did the machines have to learn how to drive, we have to also cooperate with the machines.
A lot of the accidents we see are actually people driving into the self-driven vehicles because they don't act like humans do. So that's also something that we're not used to. So there's kind of learning for humans as well, so we have this transition period which will be a bit painful, I think. We'll have to try to make the best out of it.
Now I know one of the ... During this conference you were all on a panel discussion, a very interesting panel discussion that was about roads and road markings, and lane markings, and signage. And how can we prepare our future based on that information that will have the best results for our future vehicles, and for the safety? Which is what we're all working toward. And I know that you talked about the perfect environments, and you also talked about some of the challenging environments. I think we can all picture the perfect environments with long straight roads, weather that never changes.
Let's talk about the challenges that each of you have experienced, or that your organizations are testing around. And especially with how you can picture Detroit and the Detroit region, and Michigan and the upper Midwest, of our weather. I guess what I'm getting at is our weather and our roads. What are we finding with regard to challenging situations?
Yeah, I can start. I think, you know I provide the perspective of a company that has worked for nearly 100 years, but over 80 years in transportation safety, and it's been all about making things more visible. And so that's been certainly centered around making that more visible to the human eye. What we're thinking about right now is, how do we make things more visible to sensing systems on vehicles?
There is a correlation between visibility and safety. If you can see something, you can detect it, and perhaps avoid it. So we're a big company. I represent a group in the Transportation Safety Division, which is looking at things like lane markings and signage, and better perception of static infrastructure. But there's also an opportunity too, to extend that as well to things that are moving as well. I mean, we're a company that's also...we provide materials for the automotive industry. And so for automotive applications, and so you can increase the visibility of vehicles to the sensing systems as well.
So one particular example I'll give you is, a Lidar system actually has a difficult time detecting a black freshly washed car. Well, that's a challenge that could be overcome, right, in changing how the materials are put into that vehicle. Similarly to pedestrians, I mean, your clothing may actually be masking to a Lidar sensor. And so I mean there's things like that, right, that we'll start to think about in the future as we see more and more of these systems on the road.
So I think you can bin sort of the opportunity for the infrastructure into different categories. I mean, one is, static infrastructure and extending the technology that exists right now and making that better. I think there's a connectivity piece of it too for sure, that I see that the infrastructure can provide. And then I think there's also groups looking at sensing within the infrastructure. So instead of relying on the vehicles completely to have sensing capabilities themselves, there is a certain amount of sensing that could be built into the infrastructure that could share in that responsibility. For example, in an intersection, in detecting the pedestrians that exist in that intersection and broadcasting that information.
And again, the human element. Humans are unpredictable.
Ana Dalsnes Storsaeter
Yeah. I think it's a great question. I worked with the Norwegian Public Roads Administration, and this is of course a big issue for us. We have a lot of weather, but it's similar to a lot of places in the US as well. We see snow as a big challenge, both on a road where it may cover road markings. And also interfere with Lidar or other measurements. So do we need to do something with that? Do we need to increase our maintenance perhaps? To make sure that there's no snow. But also snow tends to build up on sensors making them not work.
And we have both kind of a responsibility for making manuals on how roads should ... what they should look like. And we also plan and build roads. But we also deal with the vehicle, you know, what should the vehicle be like? And this poses a great challenge for us when we have to at some point say that, okay this automated vehicle is fit for our roads. And it's not California. This is in the north, we have snow, we have all these kinds of conditions. So yeah, it's a big challenge. I think we all need to cooperate. We need to cooperate across countries, across industries, road authorities and industry and yeah, the sensor side, everyone, we have to talk together to be able to solve this big challenge.
So let's look at the problem and let's look at what each of us can do. Technologically and also from the infrastructure side. So there is one fundamental difference between the way we drive versus the way machines drive. When we perceive the data, we perceive the 3D environment perfectly. Our brains are wired for that, eyes are wired for that. When emit sensors look at the scene, they look at the two dimensional space. And because of that what happens is when I capture the data from camera, I need multiple frames. And that's why we need perfect lane markings, we need a perfect road, so that we can see the contrast.
And contrast means not only on the road to understand the lanes, but also if somebody is crossing the road and we call them vulnerable road users, like specifically pedestrians and cyclists. Because if I am a pedestrian crossing the road, and a car is coming at 70, 80 miles an hour, my ability to maneuver is very limited. I can run, at the max. But my ability to respond to that danger is very slow. And if I have black on black clothing, or black on blue clothing, then the contrast which fundamentally a camera works on, is very low. It's very difficult for camera to spot.
And the spot-on time and do the processing and then figure out what's that. So therefore, the camera is there, and then there are other complimentary sensors. For example, Lidar that has an active illumination system. It fires the lasers and then measures the reflections and calculates the distance based on speed of light. Radar does the same thing, it looks at the radio waves and measures the reflections. And with all these sensors together you can paint a cohesive story.
So certain things, certain low hanging fruits people can do by making small changes in infrastructure. Also, I think of where we are falling short it’s somebody who is working on front sensing system, they are exclusively working on front sensing system. So one thing like what Visteon is doing, for example. Visteon is taking a more cohesive approach. So if you look at Visteon, Visteon came out of Ford. And they had been a leader in cockpit electronics, making instrument clusters. And making infotainment. But clusters are dumb display, right? They give you the gaze, they give you a speed limit and all that, what speed you are taking. Whether me as a driver consume that information or I don't consume that information, nobody cares.
Now let's forget ... So what we did is first of all we came up with a very innovative concept to connect these two domains. Instrument cluster and infotainment. So infotainment is where you have your mapping and GPS. Cluster is where you have realtime information.
So if I'm a Ford F150 driver where I'm a blue collar worker, let's say, and I have worked on the job all day. And I'm going home tired, and I'm over-speeding. And I know that on I-94 it's 5 o'clock, everybody is trying to go home. Everybody is rushing. Just by looking at these two systems without any front-looking sensing system I can fairly construct the profile of the driver, understand instead of driver trying to understand the system. System can understand the driver.
And that can bring the safety profile at a next level. Now when I pair that with an exterior sensing system that senses the exterior sensing. And if I pair these two domains together, that will be entirely different system. It will be a cohesive system. So I think in a few years we will get there, other people will get there too. But that kind of approach is needed.
So it appears that individual suppliers and automotive manufacturers are working together to come up with in-vehicle, and on-vehicle solutions for all of the challenges. The challenges of the infrastructure. But what about creating standards? Let's talk about creating standards. How will that move forward the increased level of safety on our roads, for our road markings, for signage, given that we have a global economy? That we're not just developing for a very specific area, but for the entire world.
So that's actually a great point and I brought up this topic yesterday as well, that standardization is a huge step for furthering the human development. And why it is necessary because it's solves a lot of issues. So for example what is the issue today? All these perceptions of items that are deployed for ADAS, or autonomous driving, they work on machine learning. Which means you're collect the data, you train the machine about the data that what this data is about. And then based on that, you come up with a detection of reality of the world.
Now, that data is going to vary from US to Europe, to Japan to China, to other Asian countries. If there's a standardization that the traffic sign is like that, traffic lights are mounted in such a fashion, then whatever you do here, you can apply the same learning in other parts of the world. Right now it's very fragmented. So yeah, there is a huge need for standardization, and that's where I think V2V and V2X we were talking about that yesterday, it's going to play a huge role. There is already a standardization effort going on there. There has to be more on the sensing systems as well, how people share the data. Like right now that is not open. Sharing the data among the companies is also going to be a huge step.
Okay. It seems insurmountable.
Ana Dalsnes Storsaeter
Yeah. Standardization is very important for many reasons, one of them being that things have to work everywhere. You have to be able to cross state borders or national borders and your system can't malfunction, it has to work everywhere. And that would be a lot easier if things looked the same. And if we did things in the same way.
So it is an obvious thing, but it's not an easy thing. And there is going to be a trade-off between how fast will we get things on the road, and the standardization process which always takes a lot of time. And there are a lot of interests, you know, from commercial actors or countries, or whatever.
So it's a very big issue that we need to work on. But I think it's also important not to wait for the standards to be there, we also have to test. We have to get things on the road to be able to understand how it works. And to be able to find the best solutions, because we don't know yet.
Bit of a chicken and egg situation.
Ana Dalsnes Storsaeter
Yeah, for sure.
But we have to drive the convergence because even if you look at today one small example, if you look at the cameras. Cameras are based on where we call it satellite architecture. So let's say camera is placed in windshield, you use a cable to transport the data to ECU or computer for computer processing. And what happens is when you take the data out of sensor, that is the heart of the camera, you use a simulator so that you can use as single pair of cable. There are a handful of just serializer or de-serializer companies. So you serialize the data and then when you transport it at the other end you de-serialize.
So right now the problem is if you use one kind of de-serializer the same company serializer you have to use. So now we are going towards those standardization with MIPI
alliances working on that. So it's one small example. I think we have to go in that direction. We are really far from that.
Ana Dalsnes Storsaeter
Yeah. And one way to kind of tackle this insurmountable task is, as you mentioned, a way to break it up into smaller pieces, of course. And the first step I think would be to standardize what we have today, at least in terms of infrastructure. I mean, it would help if we harmonized the way we do things across states and across countries because it will make the job for the sensor part and the car part much easier. And then we can start working on, okay we made it similar and now it can understand how we'll make it even easier for vehicles to understand the environment.
But it's complex, you have the hardware side, which you can test a sensor, you can see that it works in fog or in snow, or something like that. But the software component, it's also very challenging. Then you have to understand how does this machine learning or AI system work? And can you make sure that it will work correctly in any given situation? How do we test that? So there are a lot of issues to solve.
Yeah, I guess I'll just add to it. I think standardization is also ... You know I agree with the rest of the group here that it's really important. It's been kind of at the core of our approach in figuring out what things are going to provide a real benefit. Because typically what happens is that some new innovation is created. Because standardization takes a long time, it's an opportunity to essentially pilot that new technology and figure out what improvements and benefits it actually provides. And then in that time we're identifying essentially what the parameters around which the standardization should occur.
So by the time that particular product's been validated, we have a good idea and understanding of what the threshold for performance should be. And so again, a lot of what we look at specifically is how visible should something be? And that is something that is measurable. I think the thing that's changing is that, again, going back to the fact that we're thinking more and more about how does the machine vision system, or how does the sensing system on the vehicle perceive that particular object. And should the standards be the same now for the machine system and the human system?
Ultimately I think the answer is that they have to be somewhat complementary because there will be a mixed fleet for quite some time. And I also want to answer your question too about, what Detroit and Michigan can do to participate? And I think as was mentioned in the group too, because environmental conditions are such a challenge for perception systems and being here in Michigan where you do have a broad spectrum of different environmental conditions which includes snow, but also because you have the automotive industry right here, you have the opportunity to be out testing this stuff on a regular basis.
So I think a lot of the solutions are going to be born out of the fact that you have a lot of vehicles out trying to solve these exact problems right here in this state. And I think as we continue the dialogue between the infrastructure and the OEMs and the tier suppliers, I think we will come up with some interesting solutions. And what are going to be the best solutions for some of those challenges.
So now that you mentioned the dialogue between all of the interested parties, the stakeholders in these types of issues and developing standards, each of you has come from a unique background in that you have multi-disciplinary experience. And so you know who should be talking to whom. And that's not always happening. You will attend conferences where there are some missing bodies who maybe, for whatever reason, they aren't aware, or they don't recognize the value of their input, or can we talk about how we can ... What is it going to take to bring everyone together, those important people? And who's missing?
Yeah I think you need champions for these things too. Right? I mean, I think what we have found in our engagement with the automotive industry, is that there are particular individuals within some of these companies that share a lot of the beliefs that we have too. And it's a matter of finding that right person, right? And then establishing that and that all takes time. But I think then, it's the participation on working groups too, right? You know around the creation of standards because I think the standards bodies are interested in knowing what the AV development community and the automotive industry thinks. Especially when it comes to creating standards and policy around AV's. And so I think one way in which the automotive industry and the infrastructure community can come together is through a dialogue around standardization and participation on common working groups.
And are there any working groups?
There are some. We started a standardization group for image quality which is called P2020, it's a IEEE group. Project 2020, so I think around 300, 400 people are participating in all the meetings. But we need more. And I will build on that, so I think what is missing today is participation from policy makers. And what happens is you start looking from the gig economy, right? We're always trying to catch up because it's an afterthought. And I believe that they will again try to catch up in the autonomous field as well.
So I will go one step back and try to take a philosophical angle. So there are only two market trains, right? That feed into all of this. One is regulation, and the second one is absolute need for differentiation. You look at OEM's level, you look at Tier 1's level, you look at Tier 2's level, you also look at customer's level. We all want to differentiate. And therefore, when regulation comes into effect it drives the innovation really really far because of 10 years ago people were differentiating on rear view camera. When that became a standard, then I have to go to the next level. These regulations have to catch up. And the one way to do this is in some of these bodies, if we bring policymakers onboard. This is a way to accelerate some of these activities.
Ana Dalsnes Storsaeter
I think it's an excellent point. I think the reason it is like this is that traditionally we haven't had any needs to cooperate. Cars were made by car manufacturers and roads were made by road authorities. But now things have to work together, which is a big big change. And we need to talk to each other and that's actually why I'm here at the AutoSens conference. I was there in Brussels last year, and I think I'm the only person from the road authority side who is here both last year and this time.
And I don't understand why, personally, I don't understand why there aren't any people here from the DOT's or that sort of arena. But I guess it's just because we're used to working in our separate fields. So I think we need these arenas to meet. We need arenas to talk to each other, and that's kind of what happened.
I was at AutoSens last year, I talked to one of the organizers and he set up this panel debate today. And now you're here, and we just have to start having this open discussion about where we have to be headed. And like Abhay mentions, it's important to have the decision makers and the regulation side with you when you do this. And for us, we're wondering where the industry is going. We're not sure what the technology is going to look like. But the better dialogue we can have with the OEM's and Tier 1s and everyone, that makes us better able to understand in which way we're headed. And we can do ... I think we have to kind of walk down the road together. We can give you an indication of what our needs are, and what we want to see in the future. And then you can give us an indication, or the car manufacturers and OEM's can give us an indication about what they need. And if we talk to each other this is just much faster and easier to solve.
But we need this arenas. I think working groups are great. But they tend to be limited, they're like often at least where I'm from there's a certain amount of people who are in every working group. And it's not very effective. You don't reach a lot of people that way. I think this is important, just getting the word out there that this is needed. We often see that sometimes there needs to be some sort of publicity around it almost to get people to think these thoughts and to understand that, okay maybe I have to switch from this transportation conference that I usually go to and go to this more tech or sensor conference to understand what's going on there. And vice versa. But yeah, every arena where we can talk to each other I think helps.
So try something new, step outside your comfort level. But it's still vitally important for whatever field you happen to be in to have these types of conversations. So this is such a broad topic that we could obviously talk all day. But I'm hoping that we can finish with each of you sharing something, a nugget of wisdom that you might have learned from being at this conference that maybe you will take away with you and will be something that will sort of be bouncing around in your head, or that you want to share with others. It's a trick question.
Ana Dalsnes Storsaeter
I don't know if it's a nugget of wisdom, but one of the funny things about coming here is that when I talk to people who work with the OEM's or making systems for ADAS or AV's, they're actually the people who know someone I need to talk to at maybe your local DOT. Which is funny, because it's like Shawn said earlier, you need to know the right people. And this is a place to meet the right people. Obviously we come here to see the big trends as well as the new shiny technology that is here. But it's very important just to be able to meet someone and sit down with them and talk. And we can discover that actually our situation in Norway is quite similar to what it's like in Michigan. You know?
And I think if we leverage our knowledge and your knowledge and we start talking together then we can make this happen a lot faster, and safer. Which is the main objective.
Yeah, I think when I come to conferences like this, it's kind of a confirmation to me of the fact that there is a difference between public perception and what the insiders in the industry really think. And so I think, to Ana's point, that's why it's important for people in the transportation industry to come to things like this. Is so that they can hear the messages directly from the people who are working on the systems. Because I think maybe there's a misconception that it does take time to change the infrastructure. And that AV development is occurring and it's coming, but there are still a lot of challenges. And there is an opportunity for the infrastructure in that progression.
And through a combined approach of infrastructure improvement as well as the future advancements that are being made in the AV industry, we're going to end up with a better solution in the end.
And so I think the nugget of wisdom I would provide to the agencies who aren't at the conference is that the opportunity is here. There are things that can be done. And yes, things are moving quickly and we need to make decisions on our end as well, but we are part of the solution.
So there are few things, right? All of us are pieces of the puzzle. And very unique, right? Basically you are one piece of the Lego, and you are one piece of the Lego, I am one piece of the Lego. And when we come in the conference like this, it first of all, it helps us see how the design is coming along. And whether in that design I have to catch up, I'm going on the right track. And where industry as a whole is going. And what is my contribution? So it gives us a complete 30,000-foot view. So that's one very key item.
The second thing which is very interesting is, it helps at least me realize that all of us together, when we sit here, something that comes out of this is bigger than the sum of all of this. So basically together we are bringing more value. And why is it is because before coming here I was like at 3M. Yes, at 3M right? Basically but now I think what I can immediately think of it as when cockpit gets digitized and we go towards higher autonomy, how can we use it towards the privacy? So one small example. And there are so many of those. So I think if we meet with each other at regular intervals we can see where industry is going. And also incorporate each of those inputs.
Well Shawn, Abhay, Ana, I'm so glad that you joined me today, we've had a great discussion. And there's obviously a lot more that we could talk about. So maybe again next year we'll be able to do this again. And we'll be that much further along. Thank you so much.
Special thanks to Rob Stead from AutoSens for connecting Driven to our guests today. Join us at www.detroitdriven.us for lots more podcasts, articles, and news about how the Detroit regions leads the world in next generation mobility. I'm Claire Charlton, talk to you soon.