SIMS 7 - Assessment Concepts

George got a ‘B’ in his maths test on the website of  Online Test Provider. Miss Beale wants to get the marks for George and the rest of the class in to SIMS. Surely this must be easy!

Miss Beale would normally in SIMS:

See if there is an appropriate maths test aspect.  She will need to bear in mind that everyone else in the maths department also defines a maths test and these are usually very different as the students pass through the school.  For completeness, we will assume that this doesn’t exist.

Miss Beale would have to first decide if there was an appropriate grade set available and assuming the absence of this create one.

• Grade Set Name:  Year 4 Autumn Term Maths Test Grades (Mrs Beale)
• Date Range From: Now until <For ever>
• Grades:
  • A                  10 points
  • B                  8 points
  • C                  6 points
  • F                   0 points.

NB: The names need to be fairly specific if confusion is to be avoided.

She would then need to define an aspect (mark definition) to collect the grades.

• Aspect Name: Year 4 Autumn Term Maths Test (Mrs Beale)
• Aspect Type: Grade
• Grade Set: Year 4 Autumn Term Maths Test Grades (Mrs Beale)

Miss Beale would need to create a template which is a list of column headings for a mark sheet.

• Each column defines an aspect and optional result set (a purpose for the mark) and other optional stuff.  We only need to set the aspect to the one created above.

Once this has been done – Miss Beale would then create a mark sheet for her group and then she would be able to capture George’s mark and those of his classmates.

• At the point of entry the marks are validated and errors can easily be seen and corrected.

Once the marks are in SIMS, there is a significant range of analysis tools and reporting options that will be available to the MIS users.

Importing the marks from external systems automatically has to follow the same concepts and would be subject to the same validation. The added difficulty being that the import is often not at the point of mark entry and hence the user who might know that they meant to type a valid 'F' grade but hit the 'G' key by accident won't be available to make the informed correction.

Stage 1: Apples and Pears

We need to match up the mark definitions that we have in the VLE with the mark definitions in SIMS.  Given that the key validation point must be when the mark is entered then the VLE needs to validate the entry against the set of valid marks in SIMS before the mark is entered.

There are 2 ways of doing this fundamentally and probably many variations on a theme.

Option A: Using School defined Aspects and grade sets.

• If the marks collected are based on existing standards within the school then the correct route is to export:
  • All Aspects
  • All Grade Sets
  • [All result sets / Assessment Periods]
• From SIMS to the VLE.
• The VLE then (before marks are collected ideally) links their mark definitions to the SIMS mark definitions.

Marks are then validated on entry and then should be easily transferable to SIMS.

Option B: Using VLE supplier defined Aspects and Grade sets.

• If the VLE provided offers standard tests typically across more than one school then the owner of the mark definition should be the VLE provider.  The VLE provider would also own and manage their own grade sets and possibly result sets.
• When data is exported to SIMS, it will come complete with the VLE provider’s definitions and hence can be imported without issue.
• Each provider needs to decide which is most appropriate for them.  Typically a mix of owners is likely to be more problematic unless great care is taken.
• It is critical that if SIMS is used to generate Aspects, Grade Sets and other artefacts on behalf of a Technical Integrator then the SIMS training database MUST be reset to have unique GUID for supplier and other artefacts. 

Technical Integrators MUST contact the partner support team before generating these from training data.  Externally generated artefacts should be fine!

Stage 2 – Moving the marks from A to SIMS.

There a many ways to move the data by data we usually mean results along with their definitions.  Usually, the simplest one is to put an application on a server in the school / data centre and schedule an export/import daily. 

The key thing is that the school controls what runs and there are many excellent example of Technical Integrator systems populating their systems in this way and returning data to SIMS.

NB: Trying to push data into SIMS where the external provider is in control will usually fall foul of firewalls and security rules.

Please note that data centres may charge for hardware and software maintenance on Technical Integrator data extraction utilities. This is because it actually costs time and resources to upgrade these systems and support them within the data centre.  On a school server, this effort is usually fairly minimal and the resources are internal to the school benefitting from the application.

If the data transfer is divorced from the data entry error messages are likely to lack meaning and be difficult to get back to the member of staff who can fix the fault.    That is why there is an emphasis on validation at point of entry.

There is a check on the supplier name in SIMS assessment imports.  The customer needs to create an agency with the same as the supplier or SIMS will refuse to import the data.

 

Stage 3 - Customers using the imported data in SIMS

At this point, there are results in SIMS.  There will be no Templates or Mark sheets which are the typical way that SIMS users would view results in SIMS.  In other words, the transferred data is invisible in SIMS.

If the VLE is creating a set of aspects and they have some form of coherence when grouped in sets then it might be worth them creating templates for their aspects in a copy of SIMS modified to ensure that supplier and other identifiers are unique.  These can then be exported to an XML file and simply imported by the schools.  This is an easy task but there is no simple automation process for template imports.

If there is no coherent grouping then templates can be created by staff as in the manual mechanism described above.

In both cases, once the template has been created, it is a simple task to make mark sheets from the template.

If the aspects, grade sets and result sets are exported from SIMS to the VLE, we assume that schools would have already created suitable templates and mark sheets.

Conclusion

The value of the automation process which moves marks from a VLE (or similar) to the MIS is significant and a key step forward identified by BECTA before their demise.

It is also critical that SIMS Technical Integrators engaging in providing a facility to export data to SIMS consider this guidance carefully so that we can ensure that the quality of the data is maintained across the transaction.

Final Guidance

Technical Integrators considering writing data back into SIMS are recommended to read the information provided here which includes the details of Technical Integrator charges. Prospective Technical Integrators should also review the guidance on this site.