Last modified 13 years ago Last modified on 12/16/06 17:35:39

7. Analysis

The main research activity in this report is the analysis of whether it is possible to express advanced pedagogical models and to reuse activity constructs, resulting in pedagogical templates. Using Activity Theory as a framework we will put the emphasis on tools (LAMS, IMS LD specifications, IMS LD compatible tools and published material), XML constructs of activities and tool interfaces and community of researchers who looks into different parts of IMS LD. Referring to a modified version of the extended Activity Model of Engeström (1987) in chapter 2, our model would look like Figure 11:

Figure 11: Framework for analyzing modeling and reusability aspects of IMS LD.

This deliverable will first focus on the modelling aspect where we consider IMS LD’s ability to model case9c using the RELOAD editor (which is a reference implementation of IMS LD) and LAMS (that will soon support level A). We will also discuss several issues related to IMS LD, as well as potential solutions that have been proposed by the community of researchers. Some of these researchers have considered the conceptual model and xml constructs in their analysis.

The second part focuses on the reusability aspect where we will investigate the Reload editor, the manifest file of case9c’s learning design and LAMS express reusable constructs or components. Many researchers have also worked on the reusability aspect of IMS LD, which we will comment in more details in this report.

7.1 Modelling with Reload and LAMS

It would be out of the scope of this report to analyse different pedagogies in order to identify which concepts are central when it comes to learning and teaching. We therefore basically agree on the validity of IMS LD’s original conceptual model. The Reload editor is a reference model for IMS LD and we can use that and its manifest file to analyse how well the different parts in IMS LD express case9c. We will split our modelling in three parts: roles, activities and support environments, and the method. This choice matches the three main parts of the IMS LD concept. However, it would be a mistake to see them as being completely distinct from each other. Roles and activities are defined in Reload independent of each other and the method, but conceptually a instructional designer using IMS LD knows that roles and activities come together in the method section.

7.1.1 Roles

Every LMS must provide mechanisms for handling different roles in the system. The three main groups can be identified as students (learners), teachers and administrators. IMS LD does not specify what the actual LMS should look like and hence it does not specify the role of the administrator. However, both teachers and students are represented in our learning design.

Although case9c’s pedagogical ideas were well known, it was difficult to know which role to give the students in the Reload Editor. In particular, the Reload editor makes it ambiguous to assign a “Learner’s Role” via its interface. However, knowing that we had two main roles in this case: Student A and Student B, we therefore knew that we would have to add two new Learner roles. If the roles where not assigned in the Roles interface section, it would not be possible to assign the different activities to the different role-parts in the method section. This might be considered as an interface design problem in the Reload editor but it could also come from the way roles are handled in IMS LD.

We did not have any Staff or teacher role in case9c.

In our LAMS solution we did not have to specify in advance the roles in the course. LAMS begins with the assumption that, for a given sequence of learning activities, there is only one student. If this student is going to participate in collaborative work, we must use the group component, which can influence how the rest of the activities will take place. This means that you add the role aspect exactly when it is needed (just-in-time). However, the LAMS way of handling groups is a little problematic. In case9c we had to know beforehand how many students would attend the whole course because in case9c the students are grouped in pairs in the last section. In LAMS the number of groups can be specified, but it is not possible to either define the number of students in a group, or register students within certain groups. Also, the groups in LAMS are created randomly. The problem of randomly created numbers of students in a group will be addressed in the next version of LAMS. However, at the time of writing, it is still unknown whether the next version of LAMS will enable the teacher to assign particular students to particular groups.

In IMS LD the roles have been divided between learner and staff with certain properties. In Reload this is something a designer has to declare explicitly in the modelling process. In LAMS you only model the activity sequence for the learner. The teacher’s role is predefined as being able to monitor the sequence of activities of any learner and also as being able to intervene in some of the activities that take place (e.g. chat and discussions).

7.1.2 Activities and Environments

Each role performs an activity. This activity could either be a learning activity or a support activity and these activities produce an outcome. Creating an activity is quite straightforward in the Reload editor. You create it through the ‘learning-activity’ element within the ‘activities’ container. Each activity must be linked to an html-file (webcontent without properties) or an xml-file (imsldcontent that can hold properties). A file’s content is then considered as an activity. However, the activity is only fully defined later when it is assigned to an instructional oriented sequence of activities in the method section. One small disadvantage in the Reload editor is that the files defining the activities must be generated in an external editor (e.g. in DreamWeaver?). Once the file was created, it was possible to assign it to the activity and fill in descriptive fields. The Reload editor provides a non-graphical editor for altering text. Each activity had to specify what would happen when it was completed. This was achieved though the use of properties, either pre-defined or custom-defined. If a designer chooses to use a self-defined property, IMS LD and the Reload editor offer lot of flexibility, but it means that it would have to be created in advance and then assigned to the activity. Creating and assigning properties in a learning design is pretty tricky because IMS LD and the Reload editor only provide simple data types with predefined behaviour. It proved to be difficult and not intuitive to use these properties. A learning design editor should rather provide properties with more self- explanatory titles that give more clues to the designer on how to use them. In case9c we only used the text-property for creating a text area in the file referring to it. This file would then use set and get methods to add and view text values. Using set and get methods in a file is not trivial either, because it has to be programmed and it is not easy to keep track of reference ids of different properties.

In LAMS creating activities is different. Several activity components or tools define one or more activities. LAMS has, for example, the activity component ‘Chat & Scribe’ where students do the two activities of chatting and co-writing a document. In LAMS every activity component is purposefully designed to let students perform activities like polling, voting, writing etc. coupled with other resources. This feature in LAMS could limit the overall flexibility because the students do not have direct access to other tools. For example, an interesting activity component could be a Wiki. Currently, if a teacher wants this in his LAMS session he would have to define it as an external resource and it would be impossible to monitor the students’ activities inside the Wiki. Also, embedding a Wiki in a LAMS learning design would generate the need for single sign-on solutions (for usernames and passwords).

7.1.3 Activity flow

In the method section we connected the two students roles with the flow of activities that would take place. We had two acts: first an individual part where the students did the same activities and the second part where they paired and each student in the group had different responsibilities. In the first act we created we had to connect the two different role-parts to the same activity. This might sound strange but it was the only possibility because a student can only take one role-part within a play. During a play the student cannot change between role-parts, but can take another role-part in a concurrent play. In the second act we assigned different activity structures to the different role-parts as the students were supposed to do different and concurrently running activities.

7.1.4 Modelling issues identified by the community of researchers

There are a few researchers that have identified weaknesses in the IMS LD specification considering it from the perspective of Computer Supported Collaborative Learning (CSCL):

Caeiro et al. (2003) want to give IMS LD more flexibility and modularization of learning designs. They agree with the IMS LD proposal of Method prescribing Activities for Actors in a certain order, but they found the descriptions of these elements somewhat limiting. They use Activity Theory as a framework for understanding how collaboration in activities is tied together.

1) First of all they stress that it is problematic that an activity in IMS LD is directly connected to an environment. If they were not connected conceptually and in the xml-construction it would be easier to realize the same activity in different environments (e.g. with different learning resources). Caeiro et al's (ibid.) solution is to put the role-part at the centre of the overall model. They want to keep the IMS LD concept of act. Within an act they want to let the role-part have control of all aspects that have an influence on individual activities. In their opinion, the role-part could establish who (role) has to perform which activity (division of labour). Then (here they are departing from the IMS LD specification) the role-part should also indicate where the activity should be performed (environment) and what role the role-part has in this activity (moderator, observer etc.) Currently a role-part is fixed to one activity within a learning design but, if we were able to decouple the role-part from the environment (e.g. a Learning Object), a student could have different roles within the same activity. They also want for the role-part activity to add a definition for the outcome of that particular activity. This ‘what for’ concept could make it easier to reuse the whole activity. We agree that there is a potential problem with activities and role-parts as we experienced a similar problem with our two role-parts.

2) Secondly, they see that the method part of IMS LD becomes complicated when considering breakdown situations and learning activities that need loops. They acknowledge that IMS LD can express conditions for advanced sequencing, but it soon becomes unmanageable with complicated “if-then-else” condition statements. Based on experience from Workflow Management Systems, they suggest that acts should not be executed in sequence within a play. Instead there should be a routing condition for each act so that the role-part could go seamlessly between different acts. We can agree to Caeiro et al's (ibid.) concern about the rigid structuring of IMS LD. When a breakdown occurs, a student should be able to take action that not necessarily follows a predefined path. Although IMS LD gives the designer the ability to leave activities, acts and plays open ended, which means that the role-part can go back if needed, this cannot be forced through the design. A second point is that, to be able to add, lets say more text in a text area, the get and set properties must be changed at runtime, but as far as we know this is not possible. A comment in the WP3 Wiki emphasizes the important point that, in collaboration it is important to let students cross the borders of the predefined collaborative pattern allowing the students to go back and forth if they want to.

Miao et al. (2005) investigate IMS LD in CSCL scripting as a part on their work on the COSMOS (Collaboration Script Modelling System) tool. A CSCL script is a computational representation of a collaborative script that is used to reduce the uncertainty about coordination efforts (Mäkitalo et al., 2004), because they know how to behave and what to expect in particular situations. At present there are no authoring tools for CSCL practitioners to create CSCL scripts without a high degree of technical knowledge and designing skills. Miao et al. (2005) have studied IMS LD’s capability to formally describe CSCL scripts and experienced many problems. The pedagogical case they use is a maze script (Jansen et al., 2004). In such a script student groups try to develop strategies on how to escape out of an arbitrary maze. This case is advanced in the sense of the creation of groups and subgroups of students beforehand and at run time; the students creating artefacts at run time; students taking different roles in an activity, but then later switch roles to the same activity; changing access rights at run time and so on. This script can be applied when there is a thesis-antithesis (rule set-against-maze or pro-against-contra-argumentation) situation.

1) The first problem Miao et al. (2005) found was IMS LD’s inability to model multiple groups with the same role and the dynamic changes of groups. They found it difficult to assign a group work pattern to several groups working in parallel and how subgroups could be defined within these groups. They think the problem stems from the fact that role-parts have to be defined before run time of the learning design, but, in many collaborative scenarios, groups are formed and changed at run time. Their proposed solution is the introduction of a group element. This group role-part would have many group specific attributes not used by a person role-part. However, this group could be assigned to a role, thus solving the group/sub-group problem.

2) The second problem they identified concerns IMS LD’s run time support of artefact creation. As they write: “In collaborative learning processes, an artefact is usually created and shared by a group of people. It is normally used as a mediation to facilitate indirect interaction among group members. It may be created in an activity and used in other activities like an information-flow. For example, in the maze script the different sub groups produce, reuse and improve rule sets and maze definitions”. In IMS LD Miao et al. (ibid.) argue that an artefact can only be modelled as a property of the student creating the artefact (e.g. a vote or an answer). However, this property is not easily linked to the activity it was created in or to the other students contributing to the artefact. This could potentially be solved by the VLE using the UoL, but again this would generate an interoperability problem, as this would be only something this VLE would provide. Miao et al.’s (ibid.) solution to the problem is to introduce an artefact element. This element would hold specific values for the artefacts created in advance or at runtime such as a Maze rule set. The artefact and its status would be visible “in the environment of the creation-or consume- activities at run-time. The specification about the relations between artefacts and tools would help the run-time system to pass to or from artefacts as input/output parameters to tools automatically at run time. Some expressions and actions related to artefacts should be added for mediating group work such as get-current-users-of-artefact and change-artefact-status.”

3) The third problem Miao et al. (ibid.) identified while designing CSCL scripts with IMS LD occurred when modelling dynamic process aspects. They found that it lacked many properties for holding different values needed to keep control of the collaboration. Miao et al.’s (ibid.) solution to this problem is to add an action mechanism that allows more operations than IMS LD currently has. For example, they suggest properties to hold values for “is-all-role-members-online” and “artefact-contributors”.

4) A fourth problem they experienced was how to model complex process structures. This problem is similar to the second problem for Caeiro et al. (2003). It is difficult to model non-linear, structural relations among activities by using conditions and notifications. This problem also occurs in discussion group tools (like the knowledge building part of FLE3) where the activity (posting a message to a forum) could go on indefinitely. Miao et al's (2005) solution to this problem is like Caeiro et al.‘s (2003) and proposes to introduce transitions and routing constructs as recommended by the Workflow Management Coalition (

5) The last issue Miao et al. (2005) emphasize occurred when modelling varied forms of social interaction. They criticise the theatrical play metaphor of IMS LD where a play consists of a sequence of acts in which there is a set of role-parts doing different activities. These role-parts can run together in parallel. This metaphor does not capture the essence when “making the joint effort, people with different roles may have different rights to interact with other roles and the environment. In particular, it can not be clearly modelled by using IMS LD whether and how people collaborate, because people may work in a variety of social forms: Individually, in an informal group, in sub-groups, in a group as a whole, or in a community.” A possible solution Miao et al. (ibid.) argue would be to assign roles to activities instead of assigning them to plays. This would make it easier to go back and forth between individual and collaborative activities. This is very similar to LAMS where the designer has to explicitly define whether each activity is a collaborative one.


How can we extract pedagogical methods from a learning design based on IMS LD? Creating ready-made templates for teachers is a new area of focus within the field of educational technology and was also demonstrated in the CELEBRATE project where teachers used templates (McCormick? et al., 2004). This question is closely connected to work on the reuse of LOs (Wiley, 2002) and the more recent work on pedagogical patterns (Bergin et al., 2005). In IMS LD the idea has been to reuse the elements representing learning processes in different ways. As Colin Tattersall points out in a comment to Stephen Downes critic of reusable LOs (Downes, 2003): “Using IMS-LD it is possible both to take an existing learning design and use it with new content resources (for example applying a learning design for Problem-Based Learning to the areas of medicine, sociology, computer science etc) or have existing content resources be used with different learning designs (for example having resources on the history of Canada be used in both programmed instruction and competency-based learning approaches)”. This statement reflects much of the current work done on reuse of IMS LD’s UoL:

  • Ready made templates where the teacher fills in desired elements of an “empty” UoL
  • Reusable UoL’s where the whole unit can be exchanged between repositories and modified on a detailed level.
  • Reusable elements of a UoL where specific components (like an act or activity structure) are exchanged between repositories and modified.

First we will look at the experiment we carried out to identify the reusability of the learning design we created based on case9c. We will approach this reusability aspect within both IMS LD and LAMS. WE will analyze case9c based on the three bullets list above. In IMS LD the activity processes or pedagogical method is expressed in the manifest xml-file. This file is quite easy to read and to extract elements from. In LAMS the activity process is modelled through a highly visual modelling interface.

7.2.1 Case9c as a template

If we were going to convert case9c into a template (assuming it is a “best practice” example) on which, for example, a wizard could guide a teacher through it, we would have to introduce the ideas behind the lesson to the teacher first. This would be necessary so that the teacher would know what to fill in the different fields (although the wizard would give explanatory text for each field). Then the teacher would roughly have to run through the following three steps first:

  • Fill in introductory text for the students to read. This could be done in a text area.
  • Upload a multimedia object that would be the trigger of the student’s reflection.
  • Create some guiding questions that, together with the multimedia object, would guide the student’s thoughts about the issue being tackled.

The three steps above are the simple ones in case9c and are pretty straightforward. A narrative like that is common for many pedagogical methods where trigger information and subsequent reflection is used. The next steps are of a more collaborative nature and more complex as they force pairs of students to ask for advice and give feedback through an email service. Then the students will have to co-write a task for later submission. This part would also have to be properly described to the teacher so that they fully understand how the cross coupling of advice and co-writing of the task would work. The teacher would also have to create support material in this part. The wizard could present the following fields:

  • Fill in a problem description and questions Student A would give to Student B
  • Fill in how Student B should respond to the questions from Student A
  • Create some guiding questions that Student A should reflect on, based on the information s/he got from Student B.
  • Create some guiding questions that Student B should reflect on, based on the advice s/he gave Student A.
  • Fill in information regarding the co-writing of an essay. How the pair of students should go about doing it (writing and uploading their essay to a repository) and what they should focus on in their essay.

7.2.2 Reusable UoL

When reusing whole UoL’s like LO’s there would be problems regarding metadata for exchange and how a system could give a teacher access to modify the UoL. It is out of our scope here to discuss exchange of UoLs?.

Modifying an existing UoL is dependent on the editor’s ability to read IMS LD level A, B and C. The editor must also have a proper interface so that the components in the different levels can be modified. At level A the editor should be able to alter the method section as well as all the files referenced in activities and environment section. At level B the editor should allow the editing of properties and conditions that are assigned to the different files and at level C the editor should allow modification of notification.

In LAMS the modification of an existing LAMS course is an easy task. When you import a LAMS course (one xml file) into the authoring tool, you can instantly start editing each activity component within the design, rearrange the activity components in a new sequence, remove parts of the design or add new activity components. It is ideal for repositories that do not provide readymade “best practice” templates, but rather let the teachers themselves create a community with UoLs?. Then the teachers themselves would find the best examples of different pedagogical methods especially designed for their school or community.

7.2.3 Reusable elements within a UoL

A UoL consists of an imsmanifest xml-file and corresponding resource files. If we want to reuse only parts of a learning design we would have to look into the manifest file to identify structures that could be extracted.

Looking only at the activity element would not make any sense because, to this element, you only give it a title and assign resource files that describe that activity, the learning objectives of that activity and the prerequisites. You can also describe what will happen when the user ends the activity. This could be reused as a possible LO (content), but is unlikely to represent a teacher’s pedagogical method (Figure 12).

Figure 12: The first activity in case9c and its corresponding resource

The activity-structure element does not give any meaning as a pedagogical method either because, in this element, you only group together activity elements, which still have not been connected to role-parts. As Brouns et al (2005: p. 132) notes:

“We expect patterns to occur in the learning design method of a learning design, because the method specifies the teaching-learning process. In order to be as flexible as possible, we need to identify small re-usable components in the method section. [...] Acts are the smallest, independent sections within a method.”

We should, therefore, look at the elements in the method section where the design actually play as learning activities coupled with support activities and role-parts. We could reuse the whole method element, but that would basically be the same as reusing the whole LD. However, if we go down to a reasonable granularity level, where there is a sequence of activities connected to role-parts, we can focus us on plays and acts. Both these elements have a granularity level that makes them reusable. In our example of case9c we have only one play and two acts. As we demonstrated in the template example we should be able to reuse the whole play. And in a “big” learning design, where there are many concurrent plays, this could make sense.

We could also go down to the level of acts where many activities play together with role-parts and reuse these containers of meaningful learning processes. In case9c we had two acts: one individual and one collaborative. To reuse the individual act, we would have to extract the act-element where the different role-parts are embedded and also have to be extracted. These role-parts are connected to activity structures that consist of activities and their referenced resources. Elements for activity-structures, activities and resources would then have to be extracted as well (Figure 13). Every connection between these elements is done through an “id” property of the element itself and a “ref” property to the element it is connected to. The elements of plays, acts, activity structures, activities and resources are all at level A of IMS LD. However, if we were to reuse these elements in a learning design on level B, we would have to be able to extract all the conditions (if-then-else elements) where the activities we extracted played a part. This should be possible because all the activities that belong together in an act also have their “if-then-else” condition elements logically grouped together. These “if-then-else” condition elements also refer to their corresponding activities through their “ref” property defined in the manifest file and a “class” property referred in corresponding activities’ resource file. The “class” property is typically used to show/hide parts of a page.

When we look at reusing parts of a LAMS course, it is a little different than reusing the whole LAMS course as shown earlier. You would have to open a LAMS course and remove the parts which are not needed. But then there is no way to import parts of a LAMS course from other LAMS designs. It would have been beneficial if LAMS were able to import a sequence of multiple LAMS activity components into the sequence you were currently working on. The only way to reuse activity parts in LAMS is that every activity component in the activity toolbox can be used as many times and in as many ways that a designer wants in a LAMS course.

Figure 13: Extracted elements that constitute the act-element. There are more resources, but because of space that part has been shortened.

7.2.4 Other empirical work on templates and reuse in IMS LD

There has been a growing community of researchers working on the reuse aspects of IMS LD. They primarily look for new and innovative ways to reuse pedagogical methods that are embedded in a learning design. Usually this way of doing instructional design is based on the idea of translating theoretical ideas into a design, as in case9c. Others approach the problem differently looking at many different learning designs that through empirical evaluation identify patterns for “best practices”. There are still not many working examples of reusable learning designs except where the designers have imported a UoL and modified it to suit their needs. I will go through some of the attempts researchers have made based on the three bullets list above: templates where teachers fill in empty fields, reusing whole learning design or reusing parts of a learning design. Template tools

COLLAGE (Hernández-Leo et al., 2006) is an editor for designing collaborative learning scenarios. It is built on a previous Reload version that provided a plug-in framework for the COLLAGE tool. What COLLAGE (COLaborative LeArning? desiGn Editor) offers are templates of different learning designs. These templates have been implemented as UoLs? that can be altered and modified using the COLLAGE editor. The targeted audience is teachers that are collaborative learning practitioners (novice or not). They do not need to know IMS LD and they are not supposed to be technologists. The idea behind COLLAGE is to provide teachers with a specialized high-level collaborative learning editor that is capable of guiding teachers in the process of creating their own collaborative learning design by starting from existing templates. To achieve this, the COLLAGE team advocates the use of patterns (Alexander et al., 1977) that reflect best practices in collaborative learning. Structured as IMS LD templates these collaborative learning patterns can be applied to many collaborative learning situations. These patterns are called Collaborative Learning Flow Patterns (CLFPs) that represent broadly accepted techniques that are repetitively used by practitioners when structuring the flow of learning activities involved in collaborative learning situations.

COLLAGE offers several pattern-based templates such as Jigsaw (Figure 14), Think-Pair-Share, Simulation, Thinking Aloud Pair Problem Solving and Brainstorming. The process of selecting and authoring a template is illustrated in the following list (Hernández-Leo et al., 2006, pp. 62):

  • Choose a template depending on: The learning objectives proposed by the template, the type of problem or task the template is more suited to, and the complexity of the template in terms of the collaborative learning experience needed.
  • Read the help about the chosen template: Understand the learning flow structure on which the UoL will be based.
  • Fill in the title, learning objectives and prerequisites of the learning design.
  • Specify the collaborative learning flow: The learning flow of the selected template can be enriched replacing one or several of its phases with phases from another template.
  • Describe activities, activity completion, role-parts (including groups) and group-size limits.
  • Create the resources files needed to support the activities.
  • Assign resources to activities
  • Package the learning design into a UoL

Tests of COLLAGE show that it quite easy to do the reading, fill-in parts and packing parts if you are an experienced Reload user. However, we have not seen any proof in literature or been able of detect how to replace parts of a template with parts from another template. The COLLAGE team has done two preliminary tests with teachers that have no learning design experience and found the following limitations:

  • Teachers have no problem selecting a template and editing the title, objectives and prerequisite.
  • Teachers do not enter a description or assign resources to the activity
  • Teachers had problems understanding the pedagogical method behind the templates

The problems faced by the teachers can largely be considered interface related. The issue of assigning resources can be solved by an easy interface where it is intuitive that the teacher needs to assign resources, such as images and html files, to an activity. The problem of understanding the method can be solved by adding descriptions of the pedagogical method, which will be available in the interface. However, there are other limitations with COLLAGE that are more critical:

  • COLLAGE does not support IMS level B and C.
  • COLLAGE can only edit UoLs? that have been created by COLLAGE itself because it uses a graphical editor that cannot display all kinds of UoLs?.

Figure 14: the Jigsaw template in COLLAGE.

7.2.5 Reusing whole or parts of a LD

Reusing a whole UoL in tools like Reload is obvious as it supports all three levels of IMS LD. Most tools that are close to the specification should be able to represent a learning design, although there could be interface problems as with COLLAGE. Reusing parts of a learning design is, however, a greater issue. Currently there is only one tool as far as we know that supports the reuse of parts of a learning design. MOTPlus (Paquette, 2004) is a tool that uses a specialized type of diagram that attempts to represent knowledge in a wide variety of domains, including learning design, as a collection of meta-knowledge objects linked together by concepts such as instantiation, composition, specialization, precedence and regulation. Essentially a rich UML diagram, initially developed for an instructional engineering system, Paquette’s LICEF group, has added enhancements to enable MOTPlus to serve as an IMS LD editor.

The MOTPlus editor provides a hypertext based user interface for creating Meta-Knowledge Model Diagrams with different modes for different domains. In the IMS LD mode, they support level A constructs, including play and act. If already familiar with Meta-knowledge representation, this tool reduces the time to create learning design diagrams and the learning curve for using the tool is very short. However, if users are not already familiar with this way of representing knowledge, it could be difficult to choose the correct link type from the wide variety of available links.

To reuse parts of an IMS LD structure, a designer could simply cut an act from one learning design document and paste it into another learning design document thus reusing that specific act. That act would still keep its internal model intact.