FAQ

Will it cost more than paper-based assessment?

When you consider costs related to paper based assessments Qpercom is a cost effective solution. Clients have reported savings on costs of:

• Printing – paper, inks
• Transport – delivering paper assessments to external centres
• Secure storage facilities for paperwork
• Administration time, printing, scanning, inputting data
• Human error – mistakes made at inputting can have a huge knock on effect

With the addition of the video integration option there are also increased savings in

• Venue hire
• Travel costs
• Travel time for staff, assessors and applicants

Our admin staff are worried they will be replaced

By integrating Qpercom to replace paper based activity there is a huge opportunity for more interesting work for your administration team. Administration experts can use hours of saved time to review assessment data and support clinical leads to make informed decisions based on their assessment reports. Furthermore they might want to focus on recruitment and assessment processes for future rounds. Administrative staff are able to upskill in an increasingly digital world and you are able to evidence investment in their development. At Qpercom we offer support and training for your teams and flexible packages are available depending on your needs.

What makes Qpercom special compared with other e-assessment software?

Qpercom spun-out from the College of Medicine, Nursing and Health Sciences at the National University of Ireland, Galway in 2008. In the last 12 years we have become a global leader in OSCE and MMI software, providing bespoke assessment solutions to universities including Dundee University, Karolinska Institute and the National University of Singapore.

Our OSCE management information system, Observe, caters for clinical assessments, allowing for psychometric analysis, Excel output of all data, Borderline Regression Analysis (pre and post assessment standard setting) and support for Generalizability Analysis. Developed by medical educators, for medical educators, this web-application allows Schools of Medicine and Healthcare Sciences to plan and perform OSCEs, as well as conduct detailed instant analysis on results while the OSCE is ongoing.

We work with institutions to continually learn and grow together. We are not a static offering, if you see an area we can develop, we will work with you on the platform to achieve this.

Do I need to download anything?

Not at all, Qpercom is a cloud based solution, for both in person and our video assessment option. This means you will need only a device that can connect to the internet and a browser. Remote Assessments can be conducted across the following devices and ensure that whatever device is chosen that it has a working camera and microphone to facilitate communication:

• Desktop
• Laptop
• Tablet 

We recommend a minimum screen size of 10 inches/25.4 centimetres for comfortable viewing of all aspects of Remote Assessments. If using a device on battery please ensure that it is fully charged and will be able to last for the duration of the assessment event you are scheduled to take. The camera and device should be enabled on your device and some may have physical switches to enable. Please ensure that you close all other applications on your device to secure a stable connection at each station.

I already use Qpercom for my OSCEs, do I have to start over now you have added video integration?

No, we have added this as an additional feature which we can apply to your existing unique URL. Which will mean your existing question banks and any other data will remain. When we add this feature to your URL a new tick box will appear in your setup options, which gives you flexibility to make the assessment remote if there are last minute changes. If you would like to find out more get in touch with us.

Why is Qpercom better than Zoom, Teams and Google Meet for recruitment?

We have recently published a blog on this. In short, we offer a complete package in a browser single tab. With a bespoke experience for students, examiners, actors, candidates, assessors and of course the administrative and management teams who are already under an extreme amount of pressure due to the constant changes in guidance surrounding the COVID-19 pandemic.

Assessment score or mark sheets are on a single screen along with the video meeting. All candidates in a cohort are virtually moved between the ‘virtual assessment rooms’ which are managed by your exams team with a central control panel and dashboard. This means you are able to synchronize your assessments whilst monitoring the progress of score inputting through the exams central dashboard.

What is a Virtual Assessment room?

Imagine it like a real life assessment room/station. No one else has permitted access apart from those involved in that station. The organisation of the live video feed is triggered by your timetabling within Qpercom. Through the control panel you can virtually move your candidates or students between the various assessment rooms or stations. The assessor or examiner will ‘let them in’ when prompted by audio and visual alerts when it is time to do so.

What assessment data is collected?

You have the opportunity to import your required applicant data, this can be done through your existing system with the addition of our API option or you can do this manually as a spreadsheet (CSV) import. Results of all students and candidates are stored on Qpercom and are available to selected users. Assessors/examiners do not have direct access to overall results, however there is the option for assessors to see scores from another assessor within the station if required.

How can I use the data?

It is your decision how you would like to use the information you have available but the options are endless. The results page is fully customisable, allowing you to select and compare things like, how a candidate scored in a cohort, or compare candidates across a station throughout multiple days of assessing. At the click of the button you can organise and sort your data before you export it as a .csv file. We have Crombach’s Alpha built into the system which is a single-administration test score reliability. Commonly used among reliability coefficients.

Can I send feedback or results to my students or candidates?

Yes, all of the feedback is fully customisable based on what data you collect at the assessment. Each data field is a tick box option, and when you have selected all of your feedback parameters and candidates a pdf will be created based on that data. You then have the option to preview and customise the email or select from a custom template before sending.

Can I customise the items in the score sheets?

Yes, all of the item score sheets are fully editable. We have the following scoring options available and you can have multiple options on one sheet

• Likert – a psychometric scale commonly involved in research that employs questionnaires. It is the most widely used approach to scaling responses in survey research, such that the term is often used interchangeably with rating scale, although there are other types of rating scales.
• Visual Analogue Scale – a range can be set and a slider will appear for assessors to slide to an appropriate percentage score
• Enter Number Value –  a range can be set where a number can be entered by assessors
• Global rating score – A general judgement that can be added which will not be included in the final scores. This is to help understand if there is a very low or high scoring candidate that the intuition of the assessor matches the scores given

Will the assessors, actors or examiners see the values of each of the scores?

This is a customisable setting and can be added or removed as required per assessment sheet/questionnaire.

Where is our data stored?

Qpercom is a cloud based solution, which means you do not need to have a significant amount of space on your drivers or devices to store data. Data are encrypted and secure on Amazon Web Services (AWS Cloud service)

Who owns the data added to the system?

You do, all data added to your Qpercom solution is fully yours as Data Controllers. Qpercom acts as Data Processors and can only process data with explicit instruction from you as Data Controllers. We have a Data Protection Agreement in place with all of our clients when we sign a contract together.

Does Qpercom comply with GDPR?

Yes, we are fully GDPR compliant. Risk management of information security has never been more significant in business. To formalise our commitment to maintaining secure information, we chose to seek ISO certification. We are delighted to announce we have been awarded ISO 27001 Certification for Information Security Management Systems.

Can our staff access from anywhere?

Yes, Qpercom is a cloud web based solution, each organisation that uses our software holds a unique URL or instance. All access is password protected and permissions are customisable for each user. There is no risk of any crossover between an administrator, assessor/examiner or candidate/student as each user is authenticated and authorised when logging in and view data associated with the account type.

What browsers can we access Qpercom on?

For our video integrated solution we recommend Chrome. Chrome is free to download and easy to install. Here is the link. 

If you already have chrome installed to use Qpercom please ensure:

• Your browser is updated to the latest version
• You allow permissions within your browser for your camera and microphone

When you first login to use the Qpercom virtual assessment option you will be asked for permission to use your camera and microphone. 

We recommend that you login to do this ahead of your assessment, you will be able to do this as soon as you have your login information, this will help the assessment run smoothly on the day and help you to familiarise yourself with the layout.

Our stand alone scoring assessment solutions are accessible on all browsers that follow W3C web standards so that includes Safari, Firefox, Edge, Opera and Chrome and excludes Internet Explorer which has never followed W3C standards 

How many students/candidates can we assess?

Qpercom Recruit and Observe have a high capacity. Clients have successfully used the platform to assess several hundred students in a day.

How many administration staff can we add to the system?

All of your administrators can be added to the system. We recommend you have a system in place to ensure your staff understand individual roles and responsibilities when using Qpercom. As with other cloud based systems everything is updated in real time. We have an audit function for our ‘Administrator Users’ to see what work has been done, by who and when.

How many examiners/assessors can we add to the system?

As many as you need, we recommend you only add ones who have a confirmed exam or assessment and any who may be on standby, to make this an easier data set for administrators to manage. You have the flexibility to assign the number of assessors to stations. You can change this based on the individual station or assessment and you can decide how you want this to be reflected in the scoring. Ie. scores for multiple assessors are averaged or kept to a raw score.

Does Qpercom support Single Sign On?

Yes, Qpercom supports SAML 2.0 Single Sign On so please discuss with us how we can set this up for your organisation.

Is Qpercom integratable with our other software systems e.g. Student Information Systems, Virtual Learning Environments, etc..?

Qpercom has an API or An application programming interface. It is a computing interface which defines interactions between multiple software intermediaries. It defines the kinds of calls or requests that can be made, how to make them, the data formats that should be used, the conventions to follow. In short it means we can work with you to coexist with your existing system. If that is pre or post assessment or both.

Does Qpercom’s OSCE format work for other types of assessment?

Our clients vary in the way they use Qpercom. As well as OSCE style examinations or assessments some organisations use it to perform Multiple Mini Interviews (MMI’s) which are common interview formats. These can be used to assess different competency in each station or have a different task. We are currently expanding to offer short answer questions to our platform in an online written assessment with Qpercom Choice alongside competency mapping with Qpercom Entrust. If you are interested in a combined package contact us for more information.

Can the exam or administration team see what’s going on in all of the ‘virtual rooms’?

Yes, the administrative team has a dashboard, where they can see in real time the progress of completion of the scoresheet(s) in each individual station. Administrators can join any station live.

What if the connection is interrupted?

We know that as with face to face, virtual assessment issues can arise on the day of assessment. If there is a significant delay you have the option to reassess the station which experienced difficulties at the end. We have worked with our clients to decide the best option to manage applicants who experience difficulties and this is the preferred option. It means that the rest of the stations can continue on time and to schedule. There is also the option to pause at any point, which you are able to choose if you wish.

What if an assessor or examiner forgets to enter a score?

If a score is missed, the scoresheet will not be able to be submitted. Any questions missed will be highlighted for the assessor or examiner to go back and review.

We use competency mapping across stations. Can we see this in the results?

Yes, you are able to colour code and ‘tag’ your questions. In the results tab you are able to review the results for each tag. Results can also be sent to students in this way if required.

Some of our stations are weighted. Are we able to apply this in the final results within the system?

Yes, you have the ability to add weighting to stations or individual questions as required in your exam set up

Can I see who my hawks and doves are?

You have the ability to see in real time if there are significant differences between your assessors, you have the ability to set a threshold to identify differences between examiners at multi-examiner stations, any over this threshold will be highlighted in the dashboard for you to review. When comparing examiners at the same station across multiple circuits or sites you have the ability to view the examiners mean % and compare them against each other to see your hawks and doves.

Whilst this can happen we recommend that your interviewer training should be robust enough to avoid this occurring.

How responsive are Qpercom to queries?

We have an instant chat option available within all Qpercom assessment solutions. When you land on your Qpercom login screen you will see a chat box. If you have any questions just ask us there. We operate 9-5 Monday to Friday when we have a dedicated member of the team responding to queries with response time on average of between 3-5 minutes. If you contact us out of hours make sure you leave an email or number so we can contact you when we are next available or alternatively contact your institution.

Can the applicants/students have the same unique identifier or Identification Number that they have on our existing system?

Yes as long as the IDs are not reused for future cohorts. Qpercom uses the unique identifier to tag a candidate, if the same number is used for more than one applicant your data will be mixed.

What are the technical requirements?

Operating System Requirements

Remote Assessments can be conducted online via Qpercom’s advanced assessment solutions on the following operating systems:

• Microsoft Windows
• Apple Mac OS
• Google Android 
• Apple iOS

Browser Requirements

Remote Assessments can be conducted online via Qpercom’s advanced assessment solutions on the following browsers which follow W3C standards:

• Chrome

* Please ensure that you ensure your browser is updated to the latest version.

* Please ensure that you allow permissions within your browser for your camera and microphone. You will be asked for permission to use your camera and microphone when you first log in and you need to click on Allow when you see messages on screen

Device Requirements

Remote Assessments can be conducted across the following devices and ensure that whatever device is chosen that it has a working camera and microphone to facilitate communication:

• Desktop
• Laptop
• Tablet 

* Qpercom recommends a minimum screen size of 10 inches/25.4 centimetres for comfortable viewing of all aspects of Remote Assessments.

* If using a device on battery please ensure that it is fully charged and will be able to last for the duration of the assessment event you are scheduled to take.

* Please ensure that the camera and device are enabled on your device and some may have physical switches to enable.

* Please ensure that you close all other applications on your device to ensure a stable connection at each station.

Hardware Requirements

To use Qpercom Remote Assessments the minimum hardware requirements are:

• Processor: Dual Core Processor 
• Memory: 4GB Memory (RAM)
• Hard Disk: At least 5GB of free disk space 

Connectivity Requirements

When participating in scheduled Remote Assessments via Qpercom’s advanced assessment solutions please ensure the following:

• Connect using a wired network or WiFi connection. 
• Minimum 1.5 mbps download speed and minimum 0.5 mbps upload speed. You can check your network connection speed at https://www.speedtest.net/ 
• Disable all firewalls on your device.

* Please avoid using mobile data or a hotspot when connecting for the assessment you are scheduled to take.

Can I communicate with my examiners during a virtual assessment?

Yes, we have the opportunity for you to send notification alerts to your examiners, these are audio notifications with pop up customisable text which can be sent to all participants, select groups or individuals