High Stakes Unsupervised Testing

What we recommend

Online remote testing has changed the approach people need to take regarding testing. Since this is often unsupervised there are two critical issues that need to be addressed which are

  1. The security of the items
  2. The identity of the candidates

 

The solution we have recommended to ALL our clients for many years is a Test Identity Validation Process (see FAQ’s below). This is based on the fact that if people have the ability they should be able to replicate it. It involves re-testing candidates that make it through to a second or later stage of a selection process following screening using online tests that have been administered unsupervised.

This process requires that the second test has not been exposed and so we supply tests specifically for this purpose (our Closed Tests). These are only administered in supervised conditions thus ensuring that there is no prior knowledge of the specific items. Any person who scores significantly less the second time should be investigated since they may have had prior knowledge or help with the online unsupervised test.

Current features

With multiple versions of the tests clients can choose to vary the versions they use. Some will change the tests they use on a yearly basis; other do this more or less frequently. This reduces the test exposure and the likelihood that pre-knowledge will influence results. However there is no way to reduce the risk of cheating to zero when using online unsupervised tests. For this reason Team Focus has always recommended to its clients that they conduct a Test Identity Validation Process (see FAQ’s below). However, to reduce the risk of cheating and to increase the chances of detection Team Focus has implemented many anti-cheating measures. These include the monitoring of behaviour and timing patterns although Team Focus cannot disclose any specific measures put into place with any specific customers. Our test delivery platform – PfS – has successfully served over one million PfS online tests since 2003 to blue chip companies and government organisations.

New features and capabilities for 2019

The landscape is changing and in 2019 we are introducing additional algorithum-based test monitoring features on the Profiling for Success (PfS) online test platform. More sophisticated behavioural analysis is becoming a highly effective way to identify individual and groups attempting to cheat on tests. We are making the way that customers implement this capability to be highly configurable including flagging abnormal test results and following up on abnormal test results based upon the specific requirements of each customer. In addition, some clients would like to see test items presented in a randomised way which will be a feature available early in 2019. This feature will generate additional behavioural analysis data to identify attempted cheating even further. If required we can also provide technology solutions such as locking down remote PCs, PC Browers, and keyboards and remote screen displays to further deter and prevent attempts to cheat on online tests.

The above is an automated solution aimed at controlling the test taking environment and analysing behaviour. Some clients would prefer to use online proctoring which means that test takers are observed using video recordings. This can be done live or recorded for later examination. This adds another level of confidence in the results obtained from remote testing. Team Focus are partnering with industry leading remote proctoring organisations when it is impractical or not cost effective for early stage candidates to travel to countries and locations where local human proctoring can assure the organisation that all candidates are taking tests properly.

Team Focus invites its customers to get in touch to discuss all these capabilities, learn more about PFS anti-cheating capabilities and how to best implement them in 2019.

Team Focus Unsupervised Testing FAQ's

Can we use tests remotely (unsupervised) and yet be confident about the results?

There is no foolproof method to prevent cheating when tests are unsupervised – there is always an element of doubt. The surest way to remove that doubt is based on the principle that if someone has the ability they should be able to demonstrate that ability again. By analogy, if someone tells you they jumped over a bar 2 metres high the best way to check that they have that ability is to ask them to demonstrate. Therefore the best way with unsupervised tests is to ask people to complete an equivalent test again. This is what we call the Test Identity Validation Process (see FAQ’s below) and we strongly recommend this to all our clients (although not all clients implement this consistently).

What level of confidence can we have concerning the identity of the person taking the test?

  1. High: this requires the person to attend a centre whereby the person completes the test in a supervised environment. This is usually done at our client’s premises but can be done at an authorised centre where people can be asked to provide evidence of identity (e.g. Passport, ID card) and can also be required to have biometric details (photo, fingerprints, iris prints). Note: this is often impractical for both time and cost reasons.
  2. Intermediate to high: if testing is done remotely and there will not be a Test Identity Validation Process (see FAQ’s below) then various technological checks can be offered. Proctoring can be done remotely using 2 cameras to ensure that there is not someone else helping. This can either be Live (the most expensive) or a Recorded session which can be reviewed later.
  3. Intermediate: this involves an automated check behaviour. This is less clearly a check on an individual’s identity and more a way of identifying atypical behaviour which would suggest things like live assistance or pre-knowledge of the tests.
  4. Low: this involves administering a standard test online using secure and identifiable access.

Team Focus recommends that with all remote online testing in selection situations there should be a Test Identity Validation Process (see FAQ’s below). This corresponds to the process described above as High.

What features are available to prevent fraud in online unsupervised testing?

Clients can be offered various options as follows:

  1. Providing each individual candidate with their own unique entry codes and passwords which expire as soon as a test is started.
  2. Monitoring of on screen behaviour to identify unusual patterns such as trying to copy, download or inspect
  3. Having the test items presented in a randomised sequence.
  4. Algorithms to detect patterns that suggest a risk of fraud

How to prevent someone of creating a fake profile?

This cannot be prevented unless using the High confidence option with online proctoring (see FAQ 1). However, more commonly clients check at a later stage using the Test Identity Validation Process (see FAQ’s below).

How to prevent someone from getting someone else to take the test for them?

This cannot be prevented unless using the High confidence option with online proctoring (see FAQ 1). However, more commonly clients check at a later stage using the Test Identity Validation Process (see FAQ’s below).

How to prevent someone practicing or getting the test items in advance?

We offer practice free tests and there are plenty online. Practicing is actively encouraged (in the same way as athletes are encouraged to train). However, these are items not used in the real tests. Any online test administered unsupervised is in danger of being ‘leaked’ which is why we always strongly recommend a check at a later stage using the Test Identity Validation Process (see FAQ’s below).

Test Identity Validation Process

We strongly recommend a check at a later stage using the Test Identity Validation Process. This involves strict supervised administration using one of our Closed Tests which are not available for unsupervised testing and so the security of the items is assured.