Last week, TomTom announced the results of a University of Michigan study that “concluded that TomTom’s iPhone Navigation App and GO 2535 Portable Navigation Device provide the most accurate real-time traffic data, when compared to a number of competing products.” As I pointed out in my weekly link roundup, TomTom paid for the study (full PDF report available here), which raises questions of bias. Now we have Garmin’s take on it:
The traffic study published this week by the University of Michigan does not provide a basis for an adequate and objective comparison between traffic functionalities offered on current nüvi devices and products that compete with it. The results differ substantially from our own internal research and we believe that this is the case due to some fundamental shortcomings in how this study was conducted that challenge its fairness and objectivity. TomTom sponsored the study, and the acknowledgements specifically call out four individuals from TomTom and thanks them for their support. Garmin, however, was not made aware of the study and therefore was not able to make sure the PNDs are representative of our current technology, point out any concerns with the test plan and test environment, and to ensure up-to-date software and maps. This gave TomTom an unfair advantage as is evident from the following points:
1. The two connected Garmin nüvis in the study (1690 and 1695) were 3-4 years old (Garmin no longer manufactures those) and do not represent our latest traffic avoidance technology. Newer Garmin devices work with a smartphone app called Smartphone Link (SPL) that is designed to pair with a PND to provide a connected traffic service similar to the TomTom PND used in the study. A fair comparison would have used the SPL app instead of the older Garmin connected PNDs. The Garmin PNDs used for HD traffic, the 3490 (from 2011) and 3590 (from Jan 2012) were older models (2011/12) which have been superseded by the 3597. The TomTom PND (2535 TM) is TomTom’s latest, premium unit (see http://www.tomtom.com/en_
2. There is no assurance that the software on the Garmin devices used for the study was up to date. This is even more important considering that the devices were 1.5 to 4 years old. There is no mention of the software or map versions used on the PNDs in the study. The most current software includes any bug fixes and or map updates that might resolve problems on older devices.
3. The testing conditions do not resemble the experience a customer would have. Having six electronic devices in close proximity (see Figure 2 on page 9) can adversely affect the HD Radio reception and result in interference in receiving the HD Traffic information. Most customers would not drive with a similar set up and a fair test would have considered and eliminated RF interference.
4.The study used some navigation devices with cellular and some with broadcast connectivity to receive traffic information. It did not, however, consider how the accuracy of a traffic service is affected by signal coverage. Connected devices can only receive information where cellular coverage exists and broadcast traffic devices can only receive traffic information where broadcast coverage exists. So the decision on where a traffic service is tested can have a significant impact on the outcome. Parts of the study were conducted in areas without HD broadcast coverage, so naturally, the Garmin device with HD Digital Traffic would miss messages in these areas. This would not have been the case if only PNDs with the same type of traffic technology would’ve been used, such as a Garmin PND in conjunction with the Smartphone Link app.
Nevertheless, TomTom does have a good reputation for traffic accuracy, which may be largely due to their IQ Routes historical road speed database. But IQ Routes is never mentioned in the study. So here are my own questions:
- Was the study conducted with IQ Routes enabled or disabled?
- Where does IQ Routes data now come from? The diminishing number of TomTom PNDs on the road? iPhones?
- What kind of real world RF interference could be expected with multiple HD devices? Does an HD car radio interfere with an HD traffic receiver?
Chime in with your own thoughts and questions below…