Concerns over the use of driving test stats
ADIs are worried the use of industry data will not provide an accurate picture
Which ADIs should be prioritised for Standards Checks?
That was the questions asked earlier in March by the NASP. It followed on from DVSA changes to the Standards Check requesting systems. It came as demands on testing services by the DVSA became under extreme pressure following the pandemic lockdowns.
A lack of HGV drivers caused by Brexit and the pandemic was a key driver. This was adversely affecting supplies to shops and services throughout the country. The government made it a policy priority area in order to increase the number of new British HGV drivers and relieve pressure on supply methods.
On top of this, we all know the huge waiting times for practical driving tests.
What’s more, there are proposals to use candidate test success or failures to rank ADIs. But would this really reflect an accurate picture of teaching skills and pupil abilities?
The proposed changes by the DVSA have highlighted a lack of awareness within the industry as to how data is being used by DVSA. Along with this, there are concerns about the accuracy of in formation held and how it can be accessed and checked.
Despite these worries, the industry seem to be fairly open to the use of this type of data. However, it has to be limited and qualified.
Such data acquisition and utilisation is regarded as part and parcel of modern life and a useful tool to improve services. But there are many questions about the level of service and the quality of provision across a diverse clientele.
There are also ongoing worries that the testing system does not take effective measures to understand different needs. If performance results are published for individual ADIs, how will this reflect candidates and pupils. An ADI taking on pupils with learning or physical difficulties, high anxiety or perhaps don’t have English as a first language could be unfairly penalised.
The following are key take-aways from this survey:
- The majority of respondents were aware performance data is used by DVSA, and state they understand how it is used, but still a substantial percentage of trainers do not fully understand how data is used
- Similarly with trigger points, whilst over 50% were aware and understood of the trigger points, more than a third of respondents did not fully understand how trigger points were used
- Whilst over half of respondents were aware they could access their performance data, there was still a large percentage who do not know how to do so
- A major concern of those accessing their test data was that it was inaccurate
- The majority of respondents felt DVSA should send trainers their data to check and reflect on at regular intervals, rather than relying on trainers to request it
- There was not a high number of respondents to this survey who had been prioritised for a Standards Check as yet and, of those who had, many had not received an engagement call. Of the small number who had received an engagement call, over 60% said they did not find it helpful
- In terms of how those surveyed felt about DVSA using test data to assess individual performance over half understood the rationale but had concerns about the accuracy of data used, over a third of respondents did not think it was acceptable at all.
- In terms of a view on the regulator mandating the display of badges on test the majority of respondents had no issue with that
- When asked whether they would be happy with DVSA publishing performance data (to help the public make more informed choices) the majority of respondents were not in favour.
You can access the whole survey here.
Here are some general comments made by ADIs submitting information to the survey:
“A pupils Driving Test report does not seem to differentiate between a Physical intervention Dangerous fault and a Verbal intervention Dangerous fault. A Physical Dangerous fault is a trigger, whereas a Verbal Dangerous fault isn’t. It therefore matters!”
“The reason i would not be happy with this, is the fact that the DVSA and the general public have no idea about problems that pupils may have. This would lead to a misunderstanding and unbalanced decision. “
“There are concerns that many learners suffer anxiety and nerves which can seriously affect their performance during a test. Which may have very little to do with the trainer no matter how thoroughly and professionally they are taught.”
“A large proportion of the students that I teach suffer with either learning disabilities/mental health issues (anxiety). I don’t pick and choose my students – if I did perhaps my data would be better. I believe that everyone should have the opportunity to learn to drive. I’m already hearing of instructors telling students that they’ll never be able to learn to drive! They just need time, support and reassurance.”
“Very unhappy with the way DVSA are treating driving instructors. I totally disagree with the standards check triggers based purely on data (5 driver faults average is a trigger!!!). I feel this all leads to instructors potentially putting more pressure on test candidates to avoid us triggers, this could / does lead to more fails. It could also effect which learner drivers an instructor will choose to teach, this is unacceptable. Publishing our data also has similar problems and is unacceptable. I used to be a civil servant (for many years) and I again now feel that the DVSA is my employer, they just don’t pay me. I am a Grade A instructor but I am seriously considering leaving the profession after over 24 years
“DVSA are trying to use the test data in a way which is inappropriate and should consider carefully the negative implications. A very unfair system will be created through well intentioned naivety to how real people will react to the threat to their livelihood of having pupils who fail. The testing system then fails the very people it is aimed at helping because slow learners, nervous/anxious drivers and those with specific additional needs will be unable to find an instructor who is willing to risk their pass rate to provide a service because of the discriminatory way they will be treated by DVSA. The latest proposal are at odds with DVSA’s own motto – Safe Driving for Life as the emphasis is very clearly shifted to passing the test first time at all costs. This is far from Client Centered Learning and shifts the focus instead onto the ADI who will be judged from all sides on a 40 minute snapshot in far from everyday circumstances on the performance of an individual who’s actions are beyond the control of anyone but themselves. The average person does not perform to their normal standard of behaviour under heightened pressure (neither students nor ADI). Perhaps if DVSA considered limiting the number of attempts that someone may have at passing their L test, as with the ADI part 3, this might have a more positive impact on people only taking a test when they are actually ready? Such a move would be more supportive of and less discriminatory towards ADIs than the current proposals.”
Access and openness
“What other fields if any are the public allowed access to the performance data of individual people without their express permission? Is the data that the DVSA use an accurate and tue reflection of an individual ADI’s ability. It is well known that test passes are effected by the DVSA’s policy to investigate examiners that have pass rates below or above the national average. Is this going to effect the examiners decision to pass or fail a candidate because they don’t want to put their heads above the parapet? Are
ADI’s going to pay the price for DVSA policy?”
“The industry needs to make sure that we remain accessible to learners with characteristics that increase their risk of test failure even though they are competent outside the test environment (Thinking about a pupil of mine who had a bad experience then was triggered by reversing manoeuvres in test circumstances failing 5 times on the same thing that she could do as routine during lessons).”
“I have been an ADI for 35 years and have always tried my best to teach pupils to the best of my ability. On the odd occasion you get a pupil who suffers so much from nerves on the day of the test that they make an uncharacteristic mistake and on subsequent tests they do the same. Consequently the one or two pupils you have that suffer with high levels of anxiety on test day can really affect the instructors pass/ performance data. So in future I will stop teaching those pupils who suffer greatly from anxiety.”