I’ve had a flurry of emails and messages regarding my SCORM cheat the past few days, and have received feedback from a number of well-regarded SCORM aficionados, some of whom contributed to the standard and helped make SCORM what it is today. This is wonderful, I’m very happy to hear from everyone, especially regarding such an engaging topic.
But as I hear more from these seasoned SCORM pros, I’ve made (what I believe to be) an interesting observation: there is a sharp division between die-hard SCORM developers and casual users. I suppose I’ve felt this way for a long time, but it’s really coming into focus this week. Let me try to define the camps.
- Die-hard SCORM developers (aka scormmies). The scormmie is a person who understands what SCO roll-up means, and can hand-code an entire manifest. A scormmie thinks the word metadata is sexy. This person believes a course should be designed to use SCORM from the start, complete with sequencing and interaction tracking; if the course isn’t running in an LMS, it won’t function without being loaded into some kind of SCORM player or test suite. Scormmies get angry if their LMS hasn’t implemented the entire SCORM spec.
- Casual users (aka shruggies). The shruggie is a person who doesn’t care about multi-SCO courses. Shruggies don’t want to be bothered by the technical details, and use rapid e-learning development tools to build courses, freeing them from needing to know any of the technical mumbo-jumbo. Metawhat? “SCORM… yeah, that’s one of the publishing options in [insert product name here], right? So it will work with my LMS?”
The e-learning market has changed significantly
Over the last week I’ve mostly heard from scormmies who make comments such as ‘well, if a developer knows what they’re doing, they’d never make their course that vulnerable to begin with!‘ and ‘a developer should never design a course to only require a completion and score… that’s asking for trouble.‘
The problem with this line of reasoning is that the e-learning landscape has changed dramatically since SCORM was first conceived; the scormmie used to be the majority. Now, with the proliferation of e-learning development tools and LMSs, the scormmie is a minority. Most “e-learning developers” are not programmers by trade, and are not familiar with the very complicated and intimidating SCORM spec. They use tools that do the heavy lifting for them.
If you survey most e-learning development tools (which is a booming market), the courses they publish are almost exclusively single-SCO courses that only use the simplest core SCORM functionality: completion status, lesson location (bookmarking), score, and suspend_data. These products are designed to create courses that work without SCORM, which means they only add the minimal SCORM code needed to get the course running on an LMS; all other logic is generally handled internally. They certainly don’t use sequencing and navigation or cmi.interactions.
LMS vendors generally advise customers to buy these off-the-shelf tools to build their courses. E-learning conferences are packed with tool vendors and advertisements selling the virtues of a ‘no technical expertise required’ tool. At work I sometimes get calls from vendors trying to sell me the latest and greatest tool.
The majority of courses are no longer developed by scormmies
All of this leads to one point: I think some of the SCORM guys have lost touch with the current market and don’t realize just how much of a problem a simple SCORM cheat like mine could be. Sure, it probably wouldn’t work on courses developed by seasoned scormmies because multi-SCO courses that utilize interactions are much too complicated for my itty-bitty script to tackle… but courses developed by mainstream development tools are easy targets. Ducks in a barrel. So long as the API is JavaScript and unprotected, a script like mine can bypass the SCO completely and set the course to complete before the learner even gets past the table of contents. The only way to figure out if someone cheated is to run a completion report and look for unusual patterns, which is highly unlikely in most corporate environments. As a friend noted the other day, there are many more script kiddies who can write cheats like mine now than there were when SCORM was first proposed.
Who gets the blame for the vulnerability?
Can the tool makers be blamed? Maybe, but hey, their #1 priority is satisfying the needs of the community, and the community wants quick, easy, and ‘can run on a CD-Rom’. Could the vendors have implemented more sophisticated SCORM mechanisms? Yes. However, everyone chooses the path of least resistance (and least development dollars), and we all know SCORM development is not a walk in the park. I’ve been using SCORM for five years and still avoid most of the complicated stuff because it’s … well … complicated.
The community at large (aka the shruggies) has bought into the notion that SCORM is the standard for e-learning. This is what the scormmies wanted, and it made the most sense for everyone involved, even the tool vendors. But how many people knew about the security vulnerabilities in the JavaScript-based API? A lot: the SCORM authors, the ADL, LMS vendors, tool vendors, and a number of prominent SCORM developers. Did any of these people warn the end clients of the risks? Maybe, but I personally have never been warned of any SCORM security issues in my five odd years of SCORM work. I’ve never been told “don’t use SCORM for that because it isn’t secure.”
Why didn’t anyone act?
I wasn’t privy to the early conversations, but I’ve been told that SCORM developers have said “don’t use SCORM for high-stakes assessments” from the very beginning, circa 2000. If this is the case, why has nothing been done to improve SCORM’s security? It’s only been about nine years. Did convenience beat out security in the race to implement the standard?
I get the impression that the scormmies (and remember, my term scormmie just means a person that works with SCORM, not necessarily an official representative) felt no one would bother trying to hack the system, and that a well-built course would be so difficult to cheat that it would be easier to simply take the course. With today’s simplistic single-SCO courseware tools, I don’t think this is a valid argument anymore.
I’ve also heard from scormmies that we’re still fine, because everyone knows SCORM shouldn’t be used for high-stakes training. I think a significant number of corporate, military and government trainers would disagree with that assessment, because the LMS salesperson never mentioned it. Neither did the e-learning development tool vendor. Oh, and that instructional designer we hired out of college? She’s heard of SCORM but has no clue how it works. Isn’t it safe since you have to log into the LMS with a password? There’s a padlock icon and an https protocol… that means it’s secure, right?
Nope.
Simple-SCO courses are used for all kinds of sensitive training nowadays. Compliance training alone is huge these days and can be found in examples from almost every simple-SCO tool vendor. As a colleague recently remarked, “it’s all low stakes until someone’s attorney gets involved”.
No hard feelings!
I would like to point out that I am not targeting anyone in particular, have no animosity towards anyone, and have the utmost respect for the scormmies and what they do (I’m half-scormmie myself). I’m an optimist with a very critical eye, and this post is intended as constructive criticism… criticism intended to cause positive change.
It simply became apparent to me that at some point the scormmie community dropped the ball and got complacent; it seems as though the whole community assumed no one would bother to hack a course. Well, I did. And I used public documentation to do it. It took two hours while I was flying on an airplane, and I’m not the sharpest tack in the box. I’m sorry if my cheat script caused a stir (and if this blog post makes some people uncomfortable) but we need to talk about this issue. Now.
What’s the solution?
OK, we’ve covered enough of the criticisms and the importance of working towards a solution… I’m ready to let it rest. Let’s finish on a positive note: SCORM uses existing technology and standards, and if multinational banks can protect billions of dollars from cyber-criminals using standard web technology, we should be able to secure our courseware, too. I personally think we should be able to figure something out in the next couple of months and that it ideally shouldn’t require much work to implement — no need to wait until SCORM 2.0 comes out!
Here are some suggestions I’ve heard:
- using a secure web service to handle important duties such as processing completions and scores
- rolling up SCOs in a way that forces the LMS to analyze multiple SCOs before setting pass/fail (a second ‘dummy’ SCO could be used if the course is a single-SCO course)
- using form posts to submit the completions (the form post would contain a unique encrypted key that must match a key on the LMS)
Personally, I’m especially interested in ideas that don’t require modifications to LMS implementations and might only involve a strategic re-organizing of a SCO’s manifest or SCORM code. Perhaps using a SCO roll-up can become a security best practice, even if the course only uses one SCO? That type of simple solution would be ideal since it wouldn’t require modifications to an LMS or SCORM spec — it would only require a broad marketing effort to get the word out to all SCORM developers and toolmakers.
I would love to hear other ideas, as I feel we can probably come up with any number of workable solutions. Please add to the discussion! Remember, these need to be solutions that can be implemented easily and by the single-SCO type of courseware tools flooding the e-learning market.
By the way, while we’re at it, can we improve accessibility in our e-learning, too?
Comments
Steve Howard wrote on April 2, 2009 at 4:39 am:
I guess you and I have 'rolled' in different worlds for the last decade or so. I've contracted for loads of companies in that time, and talked with zillions of people and the vast majority of e-learning deployers I've chatted with seem to use an LMS for:-
Log on/off
Start course
Quit course
Resume course
Pass test
Complete courseWhich has led me to often ask why they even bother using an LMS …
ethan wrote on April 2, 2009 at 7:09 am:
You might want to join the SCORM 2 workgroup on api/web service dev. We are discussing the need for better security in the design right now. Pretty interesting stuff.
Philip Hutchison wrote on April 2, 2009 at 9:32 am:
@steve: that makes you a <em>scormmie</em>, my friend. 🙂
it reinforces what i'm saying: developers who've 'rolled' in SCORM circles for a long time know better. e-learning development tool and LMS customers don't. today, thanks to rapid dev tools, people no longer need a SCORM expert on their team to get a course up and running. this means they don't have a scormmie whispering in their ear about the vulnerabilities of SCORM.
if the development tool isn't built in a way that prevents these simple cheats from working, their courses will be vulnerable, and the end user has no clue. product literature doesn't say anything about it. LMS vendors (at least the ones I've worked with, including some open-source ones) don't say much, if anything.
ideally we will be able to define a simple solution that works with existing SCORM courses then spread the word to these people (vendors & customers alike).
@ethan: thanks!
Goulwen wrote on April 2, 2009 at 1:32 pm:
Scormmmies and shrummies
Hi, Philip
I totally agree with your perception of the SCORM community. In the organization I work for, I can see every day the gap between the two world, the scormmies being represented by myself while all other people clearly being shruggies.
The distinction between the two is not only true about security issues. Most of content creators are content writer, graphic designer and multimedia and interactive developer at the same time. It reminds me the beginning of the web, when the webmaster role was to develop and administer the websites, to do the end-user support and the search engine optimization.
The changes in SCORM 2.0, especially the greater programmation skills required to create services-based courses, may help the transformation of the content creator function into specialized roles, each one doing better work than the current function.
Leo Lucas wrote on April 3, 2009 at 8:06 am:
Hi Philip,
I loved this article. I have a love-hate relationship with SCORM. It does get the job done but it is way to limited and rigid in several areas. The most aggravating area for me is the sequencing in SCORM 2004. So, I submitted this white paper (http://www.e-learningconsulting.com/SCORM/LucasProposal.pdf) for the LETSI effort going on now to create a better version of SCORM. The white paper says that the LMS should NOT provide the sequencing between SCOs. The LMS has only limited sequencing options. The sequencing is extrememly hard to create (even for a smormie). And the worst part is the UI for sequencing is different for every LMS – a nightmare for the learner and author. So, the whitepaper proposes a way for a SCO to call another SCO to implement sequencing. This gives total control to the author but still preserves the reuse and data reporting for individual SCOs.
Leo
Steve Howard wrote on April 3, 2009 at 4:11 pm:
I rather think that the vast majority of e-learning doesn't use or need all the existing features of SCORM. Rather than being too rigid and limited, I think it's too bloated and complex for the needs of the majority of people in e-learning – those who only want half a dozen things tracked.
Should they need something with more features? Perhaps. But my experience has been that companies pay huge amounts to buy into Blackboard, Docent, Lotus LearningSpace (oops, showing my grey hairs …) etc because they are sold on the idea of tracking and compliance (and 'standards'), but they don't need all the whistles and bells.
And no company that I have worked with (around 40 or 50 places in the last 12 years) has employed any sort of SCORM expert… which kind of proves my point.
I think we all know that too much of the training out there is kind of crappy, and the 'rapid learning revolution' that we've endured for most of this century (!) is definitely part of the problem – companies have been convinced that any employee can create cheap training with Word, Powerpoint and "our built-in, easy-to-use quiz engine". Yeah but cheap training= crap training, right? ….
Not that I'm bitter 😉
Patrick Payne wrote on April 23, 2009 at 1:35 am:
I'm really glad that I found this blog and agree with many of the comments. I have been involved with lesson development within the defense industry for over 6 years. Mainly as an SME, but (as Goulwen said), I've also had to learn FLASH and assume the additional roles of Content Developer ISD, Graphic Artist and manager because there is a huge lack of understanding of the requirements are to build and deliver multi-SCO training lessons (involving hundreds of SCO's) designed to be run in multiple environments (Windows, Linux, Solaris) on a proprietary LMS that was ill conceived and delivered over a year after content development had begun.
Our lessons simulate a Navy tactical system for operator training. This requires our content to communicate between three browser windows using FS or local connections- stuff far beyond the capability of a few SME's. We were lucky to get the company to hire 1 Flash programmer, which was obviously born out of necessity.
We've managed to get it to work and it looks good. But there are still major issues with both the content (technical accuracy) and with the LMS. I should also include SCORM in this, because none of us really have a grasp on what SCORM means, other than the fact the our content needs to be SCO-based.
As Lucas pointed out, we quickly learned that the "content packager" that was delivered as sort of an add-on with the LMS was a hugh time and money waster for our customer and that it was much easier for us to "package" and create the SCORM manifest ourselves and therefore no longer use it. And we also raised the question of controlling the lesson flow within our Flash files, rather than sending the calls out to the LMS. Of course, this would eliminate the need for an LMS pretty much, so that didn't go over very well…
Don't know which "camp" I fit into… In some respects I guess I would be enough of a Scormmie to be dangerous, but mostly a Shruggie who has been thrown into a Scormmie role- but has no time to left on hand to study it! And even if I did, the customer or my employer wouldn't pay any attention anyway when I raised the serious SCORM issues as I found them. Same old song and dance- no funding or no time to address the issues.
That leads me into what I believe is the most serious issue we face. Management. It's not that they don't understand. It's that they don't WANT to understand! I've been raising issues like these for years now and little progress has been made to address them. So developers feel stuck and helpless to do anything to improve things and deliver better training. Complete lack of support and the day-to-day management that is required for a project on this scale is non-existent. No direction or guidance other than what we can provide for ourselves (which is limited when you have no authority or checkbook).
Managers focus on the Return on Investment only and know nothing about the projects they are managing- or even care.
Unlike Steve, I am bitter! (can you tell?)
At least I've now found some others that share our pain. Maybe through you guys I can find some other way to approach the issues we have.
Steve F. wrote on April 26, 2009 at 8:05 am:
I often wonder why 'everything' needs to be so complicated in the eyes of the SCORMMIE. I've studied the spec, I know the capabilities of a range of LMSs and I know many of the deltas in implementation for a range of packaging patterns.
Can you break things down to the LCD for optimal reuse? Yep. Will people 'actually' reuse these components? In most cases, probably not.
So why do we keep blowing that horn? Let's focus on what the users need first, what the system needs second.
The truth…
Most folks across the board, including users, data managers, instructional designers, can neither fathom nor would they ever see a need for, all of the complex object oriented (programmer world-view centric) array of assembly and deployment options available within the spec.
And I gotta say about the cheat… BFD. Fix it, don't fix it. Does it 'really' matter that a techie that knows the ins and outs of the spec was able to work a way to backdoor the system for a compartmentalized course? What's the real risk there. With all the other problems we have with the efficacy of our training solutions, and the isolated nature of the intervention (e-solutions eat alone) do we really need to be 'that' concerned about fixing 'this' piece of the puzzle?
Just sayin – SCORM standards are really technology centric. Graphics outputs are GA centric. ISD outputs are ISD background centric. The real problem isn't the aggregate outcome, it's the 'me-centric' sum of the parts…
Steve F. wrote on April 26, 2009 at 8:28 am:
Extending the previous comment –
While I believe that the depth of the technology often acts at odds with the needs of the learner and the teams that work to assemble the facilitation packages…
There IS a place for sensible roll-ups and activity focused SCO's. There's a delicate balance there, defining how much 'tech focus' (or any other 'centricity') is the real challenge. I don't think we are there and it's going to be a serious uphill battle striking that balance. Too many interest groups with dogs in the fight.
It's great, Phil, that you explicated and explained the vulnerability. It helps those that are worried about sensitive assessments (which, in my opinion, should really be facilitated anyway:)) design their assemblies to minimize potential cheating.
My point was that you were able to figure it out because you know the ins and outs. You have the keys to the city.
Of the sensitive (misplaced) assessment packages in the wild, how many users of these systems would you guess have the knowledge to pull off the cheat. There's your risk number. It's minuscule in my estimation.
BW022 wrote on March 23, 2010 at 11:08 am:
Philip,
Two issues on the articles…
Complexity
SCORM is a technical standard. If implemented correctly by LMS and authoring tools no end-user or content developer should ever need to look at the SCORM standards. I don’t need to look at FCC standards for telephones, satellite networks, etc. in order to make a phone call. A courseware developer shouldn’t need to look at SCORM in order to use an authoring tool either.
The complexity issue goes away the moment that authoring tools provide simple interfaces that output SCORM content instead of providing APIs, hooks, hard-coded logic, require manual editing of files, etc. and content developers actually use the tools. Honestly, if you want to hand-code your own HTML pages rather than use a good authoring tool… I’ll give you about as much sympathy as someone who decides to build their own telephone and then complains that the FCC standards are too complex.
Security
This is a non-issue. No online exam is *ever* secure. You can never trust the results. You can make SCORM as secure as you like (or using AICC) and it doesn’t stop a user from asking someone in the next cubical the answer. Securing the communication method does nothing to make the result trustworthy.
However, any regulated industry will know this and should have setup policies to validate competency in a subject in some manor which can be validated – i.e. a proctored exam, on-the-job skills demonstration, physically signed document, etc. And if these are in place to ensure that a friend isn’t giving you the answer… then it doesn’t matter how insecure SCORM is. Feel free to give yourself 100% on all the online exams, but it won’t do you much good since your “Pharmacy Blood Machine Usage” diploma is based on your supervisor watching you use the machine.
I guess you and I have ‘rolled’ in different worlds for the last decade or so. I’ve contracted for loads of companies in that time, and talked with zillions of people and the vast majority of e-learning deployers I’ve chatted with seem to use an LMS for:-
Log on/off
Start course
Quit course
Resume course
Pass test
Complete course
Which has led me to often ask why they even bother using an LMS …
You might want to join the SCORM 2 workgroup on api/web service dev. We are discussing the need for better security in the design right now. Pretty interesting stuff.
@steve: that makes you a scormmie, my friend. 🙂
it reinforces what i’m saying: developers who’ve ‘rolled’ in SCORM circles for a long time know better. e-learning development tool and LMS customers don’t. today, thanks to rapid dev tools, people no longer need a SCORM expert on their team to get a course up and running. this means they don’t have a scormmie whispering in their ear about the vulnerabilities of SCORM.
if the development tool isn’t built in a way that prevents these simple cheats from working, their courses will be vulnerable, and the end user has no clue. product literature doesn’t say anything about it. LMS vendors (at least the ones I’ve worked with, including some open-source ones) don’t say much, if anything.
ideally we will be able to define a simple solution that works with existing SCORM courses then spread the word to these people (vendors & customers alike).
@ethan: thanks!
Scormmmies and shrummies
Hi, Philip
I totally agree with your perception of the SCORM community. In the organization I work for, I can see every day the gap between the two world, the scormmies being represented by myself while all other people clearly being shruggies.
The distinction between the two is not only true about security issues. Most of content creators are content writer, graphic designer and multimedia and interactive developer at the same time. It reminds me the beginning of the web, when the webmaster role was to develop and administer the websites, to do the end-user support and the search engine optimization.
The changes in SCORM 2.0, especially the greater programmation skills required to create services-based courses, may help the transformation of the content creator function into specialized roles, each one doing better work than the current function.
Hi Philip,
I loved this article. I have a love-hate relationship with SCORM. It does get the job done but it is way to limited and rigid in several areas. The most aggravating area for me is the sequencing in SCORM 2004. So, I submitted this white paper (http://www.e-learningconsulting.com/SCORM/LucasProposal.pdf) for the LETSI effort going on now to create a better version of SCORM. The white paper says that the LMS should NOT provide the sequencing between SCOs. The LMS has only limited sequencing options. The sequencing is extrememly hard to create (even for a smormie). And the worst part is the UI for sequencing is different for every LMS – a nightmare for the learner and author. So, the whitepaper proposes a way for a SCO to call another SCO to implement sequencing. This gives total control to the author but still preserves the reuse and data reporting for individual SCOs.
Leo
I rather think that the vast majority of e-learning doesn’t use or need all the existing features of SCORM. Rather than being too rigid and limited, I think it’s too bloated and complex for the needs of the majority of people in e-learning – those who only want half a dozen things tracked.
Should they need something with more features? Perhaps. But my experience has been that companies pay huge amounts to buy into Blackboard, Docent, Lotus LearningSpace (oops, showing my grey hairs …) etc because they are sold on the idea of tracking and compliance (and ‘standards’), but they don’t need all the whistles and bells.
And no company that I have worked with (around 40 or 50 places in the last 12 years) has employed any sort of SCORM expert… which kind of proves my point.
I think we all know that too much of the training out there is kind of crappy, and the ‘rapid learning revolution’ that we’ve endured for most of this century (!) is definitely part of the problem – companies have been convinced that any employee can create cheap training with Word, Powerpoint and “our built-in, easy-to-use quiz engine”. Yeah but cheap training= crap training, right? ….
Not that I’m bitter 😉
I’m really glad that I found this blog and agree with many of the comments. I have been involved with lesson development within the defense industry for over 6 years. Mainly as an SME, but (as Goulwen said), I’ve also had to learn FLASH and assume the additional roles of Content Developer ISD, Graphic Artist and manager because there is a huge lack of understanding of the requirements are to build and deliver multi-SCO training lessons (involving hundreds of SCO’s) designed to be run in multiple environments (Windows, Linux, Solaris) on a proprietary LMS that was ill conceived and delivered over a year after content development had begun.
Our lessons simulate a Navy tactical system for operator training. This requires our content to communicate between three browser windows using FS or local connections- stuff far beyond the capability of a few SME’s. We were lucky to get the company to hire 1 Flash programmer, which was obviously born out of necessity.
We’ve managed to get it to work and it looks good. But there are still major issues with both the content (technical accuracy) and with the LMS. I should also include SCORM in this, because none of us really have a grasp on what SCORM means, other than the fact the our content needs to be SCO-based.
As Lucas pointed out, we quickly learned that the “content packager” that was delivered as sort of an add-on with the LMS was a hugh time and money waster for our customer and that it was much easier for us to “package” and create the SCORM manifest ourselves and therefore no longer use it. And we also raised the question of controlling the lesson flow within our Flash files, rather than sending the calls out to the LMS. Of course, this would eliminate the need for an LMS pretty much, so that didn’t go over very well…
Don’t know which “camp” I fit into… In some respects I guess I would be enough of a Scormmie to be dangerous, but mostly a Shruggie who has been thrown into a Scormmie role- but has no time to left on hand to study it! And even if I did, the customer or my employer wouldn’t pay any attention anyway when I raised the serious SCORM issues as I found them. Same old song and dance- no funding or no time to address the issues.
That leads me into what I believe is the most serious issue we face. Management. It’s not that they don’t understand. It’s that they don’t WANT to understand! I’ve been raising issues like these for years now and little progress has been made to address them. So developers feel stuck and helpless to do anything to improve things and deliver better training. Complete lack of support and the day-to-day management that is required for a project on this scale is non-existent. No direction or guidance other than what we can provide for ourselves (which is limited when you have no authority or checkbook).
Managers focus on the Return on Investment only and know nothing about the projects they are managing- or even care.
Unlike Steve, I am bitter! (can you tell?)
At least I’ve now found some others that share our pain. Maybe through you guys I can find some other way to approach the issues we have.
I often wonder why ‘everything’ needs to be so complicated in the eyes of the SCORMMIE. I’ve studied the spec, I know the capabilities of a range of LMSs and I know many of the deltas in implementation for a range of packaging patterns.
Can you break things down to the LCD for optimal reuse? Yep. Will people ‘actually’ reuse these components? In most cases, probably not.
So why do we keep blowing that horn? Let’s focus on what the users need first, what the system needs second.
The truth…
Most folks across the board, including users, data managers, instructional designers, can neither fathom nor would they ever see a need for, all of the complex object oriented (programmer world-view centric) array of assembly and deployment options available within the spec.
And I gotta say about the cheat… BFD. Fix it, don’t fix it. Does it ‘really’ matter that a techie that knows the ins and outs of the spec was able to work a way to backdoor the system for a compartmentalized course? What’s the real risk there. With all the other problems we have with the efficacy of our training solutions, and the isolated nature of the intervention (e-solutions eat alone) do we really need to be ‘that’ concerned about fixing ‘this’ piece of the puzzle?
Just sayin – SCORM standards are really technology centric. Graphics outputs are GA centric. ISD outputs are ISD background centric. The real problem isn’t the aggregate outcome, it’s the ‘me-centric’ sum of the parts…
Extending the previous comment –
While I believe that the depth of the technology often acts at odds with the needs of the learner and the teams that work to assemble the facilitation packages…
There IS a place for sensible roll-ups and activity focused SCO’s. There’s a delicate balance there, defining how much ‘tech focus’ (or any other ‘centricity’) is the real challenge. I don’t think we are there and it’s going to be a serious uphill battle striking that balance. Too many interest groups with dogs in the fight.
It’s great, Phil, that you explicated and explained the vulnerability. It helps those that are worried about sensitive assessments (which, in my opinion, should really be facilitated anyway:)) design their assemblies to minimize potential cheating.
My point was that you were able to figure it out because you know the ins and outs. You have the keys to the city.
Of the sensitive (misplaced) assessment packages in the wild, how many users of these systems would you guess have the knowledge to pull off the cheat. There’s your risk number. It’s minuscule in my estimation.
Philip,
Two issues on the articles…
Complexity
SCORM is a technical standard. If implemented correctly by LMS and authoring tools no end-user or content developer should ever need to look at the SCORM standards. I don’t need to look at FCC standards for telephones, satellite networks, etc. in order to make a phone call. A courseware developer shouldn’t need to look at SCORM in order to use an authoring tool either.
The complexity issue goes away the moment that authoring tools provide simple interfaces that output SCORM content instead of providing APIs, hooks, hard-coded logic, require manual editing of files, etc. and content developers actually use the tools. Honestly, if you want to hand-code your own HTML pages rather than use a good authoring tool… I’ll give you about as much sympathy as someone who decides to build their own telephone and then complains that the FCC standards are too complex.
Security
This is a non-issue. No online exam is *ever* secure. You can never trust the results. You can make SCORM as secure as you like (or using AICC) and it doesn’t stop a user from asking someone in the next cubical the answer. Securing the communication method does nothing to make the result trustworthy.
However, any regulated industry will know this and should have setup policies to validate competency in a subject in some manor which can be validated – i.e. a proctored exam, on-the-job skills demonstration, physically signed document, etc. And if these are in place to ensure that a friend isn’t giving you the answer… then it doesn’t matter how insecure SCORM is. Feel free to give yourself 100% on all the online exams, but it won’t do you much good since your “Pharmacy Blood Machine Usage” diploma is based on your supervisor watching you use the machine.