Last week at CES my team and I shared that we are collaborating with automakers – including the Renault-Nissan Alliance – to develop the Microsoft Connected Vehicle Platform (MCVP), a set of services built on the Microsoft Intelligent Cloud and designed to empower automakers to create custom connected driving experiences. This is not an in-car operating system or a “finished product;” it’s a living, agile platform that starts with the cloud as the foundation and aims to address five core themes that our partners have told us are key priorities: telematics and predictive maintenance, productivity and digital life, intelligent and contextual navigation, improved customer insight and engagement, and help building autonomous driving capabilities through deep learning and artificial intelligence.

The platform will serve as a reference for automakers who want to accelerate their investments in a connected car platform, and provide production-ready, globally scalable solutions that can be used as the foundation for creating differentiating experiences offered by automakers. We will build services that enable connected car platforms using Azure platform services (including our Azure IoT suite), Office 365, Skype, Bing, Dynamics, Cortana Intelligence Suite.

As we embark on building this platform, we are centered on three guiding principles:

1. Microsoft is not building a car for production – we are partnering with automakers and suppliers to enable them to build the best connected and autonomous cars possible.
2. Microsoft does not own the user experience – the user experience belongs to each automaker and should reflect their brand identity; we will build platform capabilities that enable automakers to create experiences their users love.
3. Microsoft does not own the data – the data ingested into and created by the MCVP belongs to the automaker and/or their customers, not to Microsoft. We will build services that can create exceptional value when data from multiple data sources (OEMs, suppliers, etc.) are federated together and the OEM, suppliers and end users will always be able to control what data is shared into a federated service.

Telematics & Predictive Services

Telematics are the foundation of a connected vehicle platform, providing the context of the vehicle into the platform. We are developing a high-scale telematics platform based on Azure IoT Hub, that is configured specifically for the automotive industry. This includes high-scale telemetry ingestion using our device-to-cloud (D2C) capabilities (e.g. vehicle health snapshots at defined time or distance travelled intervals such as once-per-second or once-per-kilometer), remote control functions using our cloud-to-device (C2D) capabilities – both of which we will default to MQTT as our recommended protocol – and device management capabilities including an extension of our device twin capabilities to create a vehicle twin as a topography of multiple IoT devices, and support for over-the-air firmware and software updates leveraging OMA-Lightweight M2M and OMA-DM protocols.

As telemetry is brought into the MCVP it will be persisted in a telemetry store where it can be normalized and made available for value-added workflows including predictive maintenance. One of the initial predictive maintenance sample workflows we will build will focus on predicting the condition and life of electric vehicle (EV) batteries.

Productivity & Digital Life

Automakers are all looking for ways to improve the experience for people in cars, and to connect their experiences with people’s lives outside of their car, especially as cars become more autonomous and more of your “cycles” are available in the car to be used for non-driving activities. At CES – through partnerships with Nissan, BMW, Volvo, and others – we showcased some early work we are doing in this space, including integrating your Office 365 calendar into your can and connecting it to your navigation system so that your car knows where you are suppose to be, integrating Skype for Business to make conference calls from your car more seamless and less distracting, and Cortana in the car enabling a point of presence for your digital personal assistant while in your car. We believe that technologies like this will enable a wide variety of experiences that automakers will offer that will make your life more seamless between your home, car, office and out in the world.

We are working with the new Microsoft AI group to enable Cortana to understand the automotive context and have a point of presence in the head unit (infotainment unit) so that you can have Cortana available to you at ignition-on. A potential experience was shown in the Nissan keynote (you can watch the video here). Additionally Volvo announced the integration of Skype for Business in their 90-series vehicles, and a few months ago Daimler announced they are integrating Office 365 into some of their cars.

ADAS & Autonomous Driving

Microsoft has been investing in deep learning and artificial intelligence for years – we have an AI organization that is thousands strong and have some of the best performing neural networks in the world. We will leverage the power and capability of our AI organization to build support for automakers who are investing in autonomous driving capabilities – from automatic emergency braking, to lane keeping, to fully autonomous L5 driving. This starts with our deep learning toolkit. The Microsoft Cognitive Toolkit—previously known as CNTK—empowers you to harness the intelligence within massive datasets through deep learning by providing uncompromised scaling, speed and accuracy with commercial-grade quality and compatibility with the programming languages and algorithms you already use.

We will build deep learning assets for training ADAS and autonomous driving systems and enabling cars using these trained models to continuously get more intelligent with OTA updates of revised models based on retraining with newly acquired data collected through ADAS systems on the road. This creates a virtuous cycle of connected cars leveraging our deep learning and artificial intelligence capabilities and continuously improving by acting as data harvesters contributing to an continuous learning system.

Our goal is not to be a provider of autonomous driving systems, nor to (as our principles state) build an autonomous car, but rather to partner with the providers of in car sensors, sensor fusion and autonomous drive systems and improve their ability by leveraging the scale of our AI investments.

Intelligent & Contextual Navigation

In December we announced our extended partnership with HERE to be the base map provider for Bing Maps and to extend the use of their maps for automotive use. This not only enables the use of Bing Maps in the car, but also the use of Cortana for location based data (Cortana uses Bing Maps for local search). We also announced a strategic partnership with TomTom as the first (of multiple) map providers we will partner with to build a set of location based services in Azure. We also announced a continued partnership with Esri as a GIS provider that will compliment these services.

Our intent is to build a World Graph of devices and objects, their locations, and how they are interconnected. This is not only a high definition map of the entire world, but a constantly changing graph of relationships of things and their place and context in the world. The World Graph will enable new experiences both in and out of the car, and will be capable of provide connected cars with high definition maps for autonomous driving.

Where the deep learning systems we will use to create more intelligent ADAS and AD systems (the reactionary systems used in an autonomous car), the World Graph will provide the proactive contextual information about where a car is, where it is going and the rules of engagement between the two points. Our intent is to eventually achieve a level of capability where we can represent objects in the world to a 10cm accuracy or better, and reflect changes in the physical world in under one-minute. This will take time to achieve and will only be done through partnerships with automakers who will feed realtime data from cars into the World Graph and get valuable location based services out in return. These services include not only strategic maps and routing (routes from A to B) but also tactical details providing lane-level detail speed limits, hazard indications and more. As cars feed data into the service, they get more accurate data back out, effectively crowd sourcing information about the world.

Customer Insight & Engagement

One of the common themes we heard across the automotive industry is a need for better insight into their customers and better avenues for engagement. When you look at all of the data that is collected into the MCVP through telematics, productivity, and the World Graph you can see that this creates a big data opportunity. Throughout Microsoft we have significant investments in analytics and customer engagement that we can bring into the MCVP. Whether it is through our AzureML and Power BI tools or though Dynamics CRM and ERP offerings, we have software that can be leveraged to generate actionable intelligence for automakers and improve their ability to directly engage with their customers.

Some of our early investments are in analytics to support the R&D aspects of new car development and give automakers insight into their fleets of test and certification vehicles, reducing the time and complexity to understand how their fleets are performing in this critical phase. We are also investing in the development of cross-platform applications to give drivers, owners and users of vehicles real-time insight into their vehicle, from its current fuel level, tire pressure and battery level, to historical trends and predictive perspectives about how their vehicle has and will perform.

Summary

In the coming weeks I will dig into each area and provide more technical detail into what the Microsoft Connected Vehicle Platform will provide to our automaker partners. Until then you are invited to read this white paper and look at this infographic.