Aligning learning outcomes and assessment methods: a web tool for e-learning courses
Gil-Jaurena and Kucina Softic International Journal of Educational Technology
in Higher Education
Aligning learning outcomes and assessment methods: a web tool for e-learning courses
Inés Gil-Jaurena 1
Sandra Kucina Softic 0
0 University of Zagreb, University Computing Centre - SRCE , Josipa Marohnica 5, Zagreb , Croatia
1 Department of Theory of Education and Social Pedagogy, Faculty of Education, Universidad Nacional de Educación a Distancia (UNED) , Room 220, C/Juan del Rosal 14, Madrid 28040 , Spain
A learning outcome approach to teaching and a learning-oriented assessment are increasingly framing higher education. In an attempt to facilitate professors' teaching activities, the TALOE (Time to Assess Learning Outcomes in E-learning) project has developed a web tool that aims to help trainers decide on the assessment methods to use in their online courses. Based on a conceptual model (ALOA) built upon Bloom's revised taxonomy of learning objectives of the cognitive domain, the web tool automatically aligns learning outcomes stated by a user with recommended assessment methods. The web tool has been developed with the European Commission support within the Lifelong Learning Programme and it is publicly available. This paper presents the background and description of the tool and the results of the testing, where various stakeholders have provided feedback about the easiness and friendliness of the tool, as well as about the utility and quality of the recommendations it provides. Prospective of the web tool, limitations and strengths are highlighted in the paper.
Learning outcomes; Assessment; Web tool; e-learning
The Bologna process and the implementation of the European Higher Education Area
brought a need for a substantive change in the pedagogical model in
higher education, requiring improvement of the traditional ways of describing
qualifications and qualification structures, modules and programmes. The focus moved from
the mere accumulation of knowledge to skills acquisition by students. The result is a
shift from a content-based approach to a learning-centred approach, a change from
teaching objectives to students’ learning outcomes. Learning outcomes are becoming
fundamental for structuring the standards and guidelines of quality assessment in
higher education and continuing education institutions in Europe and worldwide. In
this context, the assessment of learning outcomes becomes a crucial process in the
With information and communication technologies increasingly used in education,
new opportunities for improving teaching, learning and assessment are arising. Time
to Assess Learning Outcomes in E-learning – TALOE (http://taloe.up.pt), is a project
financed by the European Commission from January 2014 to December 2015 (Ref.
543097-LLP-1-2013-1-PT-KA3-KA3MP), which intends to promote the internal
consistency of online courses. The main goal of the international project TALOE is to
develop a web-based tool to help teachers and trainers decide on the e-assessment
methods to use in their online courses.
To be able to develop the practical tool, the TALOE consortium1 has set and worked
on the following specific tasks:
1. Research and selection of innovative e-assessment practices that take advantage of
the use of technology.
2. Development of a web-based tool that is easy to use by the stakeholders.
3. Testing of the implementation of the tool with real case studies.
4. Distribution and dissemination of the TALOE tool among stakeholders.
This paper focuses on tasks 2 and 3, i.e. development and testing of the TALOE web-tool.
The first task, selection and analysis of innovative e-assessment practices, was
leted during 2014
and has been addressed in previous publications
Gil-Jaurena et al. 2015)
. It consisted on the collection of 18 cases of assessment practice
in online courses from different institutions around Europe, mostly from the project
partners. They were selected according to a set of criteria, built upon the Standards
and Guidelines for Quality Assurance in the European Higher Education Area
and some principles agreed within the project partnership that the selected
eassessment practices had to meet:
Selected practices should develop assessment of more holistic, complex activities
using knowledge and skills in problem-solving or authentic tasks.
Selected practices should use a diverse range of assessment methods, resulting in
qualitative descriptions or judgements and where assessment is often integrated
with teaching and learning and may involve students as active participants.
Selected practices should identify and describe achievements according to relevant
criteria and standards.
Those principles refer to “authentic” assessment with a learning-oriented approach
(Rodríguez-Gómez & Ibarra-Saiz, 2015)
. The goal of this selection of case studies was
to obtain examples that can be used as a showcase of current practice and also as
testing material during the second year of the project (as will be explained in a later
section). It as well enabled the insight how much teachers acquainted with learning
outcomes and about the relation between learning outcomes and assessment methods.
This selection also provided information about assessment methods that are usually
used in online courses. The case studies are written on predefined templates and detail
information about the learning outcomes, course content, teaching practice and
assessment methods. In addition, a separate section was developed to linking the assessment
to the selection criteria.
Theoretical framework: the ALOA model
Not all assessment methods are valid for each type of learning outcomes. The TALOE
project uses an existing framework called the ALOA model (Aligning Learning
Outcomes and Assessment) (Falcão 2013). This model highlights the connection
between the intended learning outcomes and the assessment strategy used during a
course. So, it provides tools for linking learning outcomes and assessment tasks.
The ALOA model was developed by
from the concept of alignment
(Anderson & Krathwohl, 2001; Biggs & Tang 2007; Bloom, 1956; Boud & Falchikov, 2006)
which means that the learning outcomes of a course should be used to define the teaching
and learning activities and the assessment tasks. The main principle that supports the
ALOA model is that, in order to ensure the validity of assessment in relation to what is
intended from a course, it is necessary that the outcomes measured by the assessment tasks
are the same as the ones expressed in the learning outcomes. In this sense, the ALOA
model matches the basis of learning-oriented assessment (Carless, 2007), “an attempt to
emphasise the learning features of assessment and promote their development” (p. 58).
The ALOA model uses the revised version of Bloom’s Taxonomy of the cognitive
(Anderson & Krathwohl, 2001)
to establish the link between the learning
outcomes and general assessment methods. Thus, the model has three components:
a) The learning outcomes:
They are “statements of what a learner is expected to know, understand and/or
be able to do at the end of a period of learning”
(Ministry of Science, Technology
and Innovation 2005, p. 29)
. A learning outcomes approach has become
fundamental in the European Higher Education Area (EHEA), the so-called
The revised Bloom’s taxonomy of educational objectives
(Anderson & Krathwohl,
includes six verbs representing a continuum of increasing cognitive
complexity: remember, understand, apply, analyse, evaluate and create. 19 specific
cognitive processes identified by these authors in relation to each verb are used in
the ALOA model to state learning outcomes (see Fig. 1 and Fig. 3 as an example).
b) The assessment methods:
Based on the work of
Brown et al. (1997)
, the model includes a selection of six
categories of general assessment methods:
Multiple choice questions (MCQ)
Short- answer questions
Reflective practice assignments
Each of the categories is divided into more specific assessment methods. In the
TALOE project, each subcategory was then matched to specific e-assessment
practices (Soeiro et a
) (see Fig. 4 as an example).
c) The alignment:
It is the relationship between the first two components. A course is aligned or
consistent if the description and classification of the learning outcomes and the
assessment tasks match.
The TALOE project intends to apply the ALOA model to the specific context of
Development and description of the TALOE web tool
The TALOE web tool (available at http://taloetool.up.pt) is being developed to help
teachers and educators (users) decide on the assessment strategies that will be used in
their online courses. It provides users with a possibility to analyse the provided courses’
or modules’ learning outcomes, and offer the most appropriate e-assessment methods
consistent with the intended learning outcomes. The tool is envisioned to be used by
teachers/faculty/trainers, either to check if the existing assessment methods of their
course or module are consistent with the stated learning outcomes, or to be advised on
the most appropriate assessment methods for the existing or new course or module.
The TALOE project partnership produced a first version of the web-too
l at the
beginning of 2015
. At that point it was decided by project partners to simplify the tool
procedures during the first phase of testing due to the complexity of the ALOA model
(Falcão 2013). The present version of the TALOE web tool does not discriminate
between knowledge types, as suggested by the theoretical model. After initial testing was
completed (phase 1, explained in the coming section), it has been confirmed that the
setup matrix is working properly. The best (most appropriate) e-assessment methods
are selected on the base of the absolute matches between input (learning outcome)
and the e-assessment methods.
The welcome page of the TALOE web tool (Fig. 2) explains how to use it.
In the first step (Fig. 3), the user describes the learning outcome that students should
achieve in the course or module. The learning outcome should be described in a clear way
and kept simple. For example: Apply algorithms based on prime numbers on practical
Basic information about writing learning outcomes and the ALOA model are
available within the TALOE web tool.
After describing a learning outcome, in the Step 2 the user chooses up to three verbs
that best describe it (Fig. 3).
This step also enables the user to check and review the stated learning outcome. After
describing the learning outcome with verbs, the user sets the process in motion and receives
an assessment advice for the defined learning outcome (Fig. 4). The received outcome
provides a list of potential e-assessment methods to use, with the corresponding description.
A specific section in the TALOE website provides information and examples about
The decision engine consists of the estimation of the score that measures the best
match between the cognitive processes submitted by the user and the specific ones of a
given assessment method. The assessment methods are selected on the basis of score,
calculated as the ratio between the number of matches between input, and the model
assessment over the number of the selection of the assessment method.
Based on the received inputs, the tool calculates the best assessment methods
for the defined learning outcomes. For each suggested assessment method, a
description is provided to give users a better idea on how to use and integrate it in
Therefore, the TALOE web tool can be used in two ways: to check if the existing
assessment methods in a existing course are in line with the stated learning outcomes;
and to help users decide on the most appropriate assessment method for a new course
or module. The TALOE web tool can be used to learn more about learning outcomes,
assessment and e-assessment, as it provides guidance to teachers to formulate the
learning outcomes in accordance to Bloom’s revised taxonomy
(Anderson & Krathwohl,
Testing of the web tool
One of the commitments and tasks of the project was to test the implementation of
the web tool. In this section, we present both the testing method and the results.
The dimensions considered in the evaluation of the tool were:
Easiness: The TALOE web tool is easy to use
Friendliness: I can use the TALOE web tool without any written instructions
Usefulness: The TALOE web tool is useful
Skilfulness: I quickly became skilful with TALOE web tool
Quality: The TALOE web tool provides good, quality results
Agreement: The provided results are in agreement with my plan
Validity: The TALOE web tool enabled me to better align assessment methods to the
Help to define assessment methods: The TALOE web tool helps me understand how
to define the assessment methods better
Recommendation: I would recommend the TALOE web tool to a friend
Changes: What features would you change?
Missing aspects: What features would you add?
Removable aspects: What features would you remove?
These criteria were agreed among project partners after some working sessions
(faceto-face in the project meeting that took place at the University of Padova (Ita
, and virtually in the audio meetings we have held monthly). The instrument
used was an evaluation form, which included nine close-ended questions (Likert scale)
and three open-ended questions. It was first piloted within the project consortium
(phase 1); annex 1 includes the evaluation form used in that first phase. Based on the
results of phase 1, the web tool was improved and converted into a web form (phase
2). We used Google forms to collect the information.
The structure of the evaluation form was the following:
– General information about the course/module/lesson: course name, scientific field,
programme level, number of credits.
– Evaluation of the TALOE web tool (nine close-ended questions – Likert scale, from
1-totally disagree to 5- totally agree). The questions are above in italics in the list of
– Suggestions for improvement (three open-ended questions)
– Demographical questions and contact information (optional) about respondents:
institution, country, academic profile.
The evaluation of the TALOE web tool was done in two phases.
1) First phase testing was done with the 18 collected case studies collected in the first
task of the TA
LOE project (Lõssenko, 2014
, as explained in the introduction) and
performed by the project partners to see if the suggested e-assessment methods are
closely related to the defined learning outcomes. For each case study one learning
outcome was chosen and described with three verbs of the revised Bloom taxonomy
(Anderson & Krathwohl, 2001)
, and then checked with the TALOE web tool for the
e-assessment method. Learning outcomes in each case study were chosen randomly.
Partners encountered some difficulties when trying to describe a learning outcome
with three verbs, as some learning outcomes were not clearly defined. Another
difficulty encountered in this testing was that some teachers did not clearly correlate
specific e-assessment methods with the learning outcomes, and it was difficult to make
comparisons between planned (by teachers) and suggested e-assessment method (by
TALOE web tool) in such cases. The information was used to improve the web tool
and its usability, and to improve the evaluation form itself. This phase took p
2) In order to perform the second phase testing, the evaluation form was added to the
TALOE web tool site to make it available online. Second phase testing was done
with invited stakeholders. Each project partner invited at least four stakeholders to
test the TALOE web tool. Professors, researchers and heads of quality or innovation
departments were approached for the testing. In total 42 participants (stakeholders)
evaluated the tool. The sample is described in the next section. Feedback was
collected from mid Apri
l to mid June 2015
The participants in the online testing could remain anonymous, as demographical
questions were optional, except the question about the home institution. Nevertheless,
the majority of participants left their contact and wanted to receive further information
about the TALOE project and web tool. This was important since it reveals the
relevance of the TALOE project results to the stakeholders and helps the Project
Consortium build a database of stakeholders for dissemination and exploitation activities.
The profile of the 42 respondents (i.e., stakeholders invited by the project partners) to
the evaluation form in phase 2 is presented in Figs. 5, 6, 7, and 8.
Different scientific fields have been represented among the web tool testers: 38 %
come from the field of Education, followed by 19 % from Computer Science (Fig. 5).
Twelve percent come from Humanities and another 12 % from Social Sciences. Seven
percent work in Health Studies and another 7 % in Mathematics.
More than 92 % have tested the TALOE web tool in relation to a higher education
level course (Fig. 6). Forty-five percent have tested it in a Bachelor or Undergraduate
level course; 31 % in a Master or Postgraduate level course; 14 % in a continuing
education level course.
Fig. 8 TALOE web tool testers’ academic profile
Testers of the web tool work in 11 different countries. Most respondents are from
Hungary (24 %), Spain (21 %) and Portugal (14 %). All the project partners’ countries
are represented in the sample, plus some stakeholders from other regions not directly
involved in the project, such as Germany, Ireland, Finland and Serbia (Fig. 7).
Almost 80 % are professors and/or researchers in a higher education institution
(Fig. 8). Thus, they have direct and practical experience in defining learning outcomes
and deciding about assessment methods in higher education.
Besides this demographic information that was included in the feedback survey, given
the home institution we can state that most respondents (specially those in higher
education) have experience in e-learning: some of them work in distance education
universities and others use a blended learning mode.
In the next section we present the findings related to each dimension considered in
the evaluation of the tool, as explained in “method” section.
The testing done in the first phase (with the TALOE case studies) showed that the
majority of the teachers have defined the appropriate e-assessment methods for the
learning outcomes. In some cases, the TALOE web tool suggested additional possibilities in
terms of e-assessment methods. For several cases the TALOE web tool indicated that
the existing assessment methods should be revised.
The analysis of received feedback in the second phase (through the web form) from 42
participants is presented in Figs. 9, 10, 11, 12, 13, 14, 15, 16, and 17 and explained in this
section. The legend in those figures ranges from 1 (totally disagree) to 5 (totally agree). In
the interpretation of results, for calculating percentages we have added replies 1 and 2 as
disagreement and replies 4 and 5 as agreement. We have considered reply 3 as neutral.
The majority of participants found TALOE web tool easy to use (Fig. 9). This is an
important feature as it influences teacher’s attitude towards the use of the web tool.
Answers to the question about use of the web tool without written instructions are
distributed along all ranges (Fig. 10). The majority of participants (64 %) did not need
any written instruction to use the tool. This indicates that the tool is well designed, but
also that participants may be experienced in defining learning outcomes and familiar
with the revised Bloom taxonomy. Nineteen percent of participants needed additional
instructions to be able to use the tool.
Although some participants needed instructions to use the tool (Fig. 10), the majority
of respondents found it is very useful (Fig. 11), which is very relevant for the project.
Some participants (17 %) did not have an opinion about the usefulness of the tool. This
perception can depend on their knowledge and experience in defining learning
outcomes and the extent to which they have been achieved. Those more experienced may
find it not as useful as they are quite experienced in writing learning outcomes and
linking them with teaching and assessment.
Eighty-six percent of respondents found the web tool easy to use or become quickly
skilful with it after first use or by reading the posted instructions (Fig. 12). This result
is relevant for the project, as it compensates to same extent the difficulties that first
users may experience (Fig. 10).
Results provided by the TALOE web tool, i.e, recommendations about assessments
methods for each learning outcome, are of good quality for most testers (Fig. 13). Some
participants (26 %) could not conclude about the quality of the received results.
The majority of participants (81 %) found that received results are in agreement with
their own assessment plans (Fig. 14). Only 5 % of respondents found that the
assessment methods recommended by he TALOE web tool don’t match their methods. This
is an interesting result that would require further research: at first sight it suggests that
professors are correctly aligning their learning outcomes and assessment methods, and
that the TALOE web tool identifies this alignment.
In relation to the previous question (Fig. 14), most respondents state that the tool helps
them to better align learning outcomes and assessment methods (Fig. 15). It could mean
that the tool reinforces the assessment plan they were already using (given that there is a
high match between the recommendation provided by the tool and the existing plans,
Fig. 14). Almost 30 % of respondents don’t have a clear opinion about this question.
The majority of participants (76 %) find the TALOE web tool can help them to
understand how to better define the assessment methods in their courses (Fig. 16).
Finally, the majority of participants would recommend the TALOE web tool to a
friend; still, there are some of them (14 %) who are not sure about it (Fig. 17).
The information collected from the stakeholders in the open-ended questions of the
TALOE feedback form provides some clues about missing elements or interface
recommendations that could help improve the tool. From the 42 respondents, 23 have
submitted their comments about changes needed in the web tool, missing aspects and
Regarding changes in the TALOE web tool procedure or interface, some relevant
feedback is present in the following quotations:
“The explanations are organized along a two-level description so they are difficult to
find” (tester 3)
“Separate steps 1 and 2 (otherwise, people will look into the options of step 2 before
filling in step 1) - I did it” (tester 19)
Fig. 17 Recommendation of the TALOE web tool
“Possibility to choose more than 3 verbs and split into parts of LO could be useful”
About missing aspects in the web tool, we could highlight a demand for examples of
use of the different assessment methods, expressed by the testers in the following terms:
“There should be more detailed descriptions of methods, examples of use” (tester 4)
“Perhaps include some examples to better explain how to relate assessment methods
to outcomes and implement them” (tester 17)
“I would some examples of the types of assessment. For example for multiple choice,
one or two examples of different MCQ. Add advises how to construct a good question
for essay or MCQ” (tester 25)
There are not remarkable removable elements addressed by testers.
In summary, the feedback shows a positive evaluation regarding the usability of the
tool and the quality of the output/results, i.e. the e-assessment method suggestions.
Invited users found this tool easy to use and useful, but would like to see it improved for
more complex testing. These results also confirmed that the web tool provides support
and guidance to teachers to formulate the learning outcomes in accordance to Bloom’s
revised taxonomy, as intended and planned by the task. It also increases the accuracy
of e-assessment methods received by the tool and the alignment between learning
outcomes, assessment techniques and teaching methods.
Discussion and conclusions
The European project TALOE has successfully transformed the ALOA model into a web
tool that is public and freely available to any interested user. The web tool helps users to
align their intended learning outcomes in a course (defined using the Bloom’s revised
taxonomy of learning objectives in the cognitive domain) with appropriate assessment
methods. After a period of testing of the tool, the evaluation results presented in this
paper report that the tool is easy to use, written instructions are helpful for users, the tool
is useful for its purposes and provides valuable recommendations. The requested
improvements that the TALOE project will try to address are related to two main aspects:
– Improving the interface, usability and instructions. These changes have been taken
into account to some extent in the last version of the web tool, which includes
information about how to better write learning outcomes, about the ALOA model,
defines the diverse types of assessment methods and provides some examples of
assessment methods in the field of e-learning. The project consortium is also
considering the translation of the web tool into languages other than English, in
order to improve usability.
– Increasing the complexity of the web tool, thus providing more complex
recommendations, especially examples of use of the diverse assessment methods. Users
need more information about how to move beyond the identification of appropriate
assessment methods to the implementation phase. The tool will be further developed in
phases of complexity of the definition of the procedures relating to the learning
outcomes and assessment methods. This is an extra effort to achieve consistency of the
web-tool performance and simplicity of procedures by potential users.
Besides this prospective that the TALOE project wishes to achieve, we would like to
briefly highlight some concluding remarks about strengths and limitations of the
TALOE web tool.
A main limitation that users ought to be aware of is that the tool is focused on the
revised version of Bloom’s Taxonomy of the cognitive domain
(Anderson & Krathwohl,
(Fig. 1), thus it does not contemplate other learning domains, namely the
affective and psychomotor domains
(Bloom et al. 1964; Kennedy et al. 2006)
, and the
social domain of learning
(Vygotsky, 1978; Wenger, 2000; Garrison & Anderson, 2003)
It is important to notice this limitation, so users can be aware that not all the learning
outcomes they state for their courses would be represented in the TALOE web tool
(but only those in the cognitive domain).
Among the strengths, we would like to highlight the value that testers of the tool
have stated and the fact that they would recommend its use to their colleagues. In the
basis of the European projects it is the dissemination and exploitation of the developed
products. In this sense, we feel encouraged to share the TALOE web tool with any user
interested in checking if the existing assessment methods in an existing course are in
line with the stated learning outcomes, and in making decisions on the most
appropriate assessment methods for a new course or module. An extended use of the TALOE
web tool could lead to another research project, focused on analyzing the impact of the
tool on the improvement of e-assessment and, furthermore, on exploring the impact of
the use of the TALOE web tool on the improvement of learning, which is, at the end,
the purpose of the educational experiences we design and develop.
1The following partner institutions comprise the TALOE consortium: Universidade do
Porto (Portugal) (coordinator); Gábor Dénes Főiskola (Hungary); Sveučilišni računski
centar Sveučilišta u Zagrebu (SRCE) (Croatia); Innovate4Future – Center for Advanced
Educational Solutions (I4F) (Romania); Università degli Studi di Padova (Italy); European
Distance and E-Learning Network (EDEN); European University Continuing Education
Network (EUCEN); Hariduse Infotehnoloogia Sihtasutus (HITSA) (Estonia) and
Universidad Nacional de Educación a Distancia (UNED) (Spain).
Inés Gil-Jaurena holds a PhD in Education and is an Associate Professor at the Faculty of Education, Department of Theory
of Education and Social Pedagogy, National Distance Education University (Universidad Nacional de Educación a
DistanciaUNED), Spain, where she has been teaching and researching since 2005. She has been involved in various European
projects related to open and distance education.
Sandra Kucina Softic hold a Master’s Degree in Digital Education by the University of Edinburgh and is Head of
the E-learning Centre and Assistant Director for Education and User Support at the University Computing Centre - SRCE
University of Zagreb. Previously, she was Coordinator for Development and Implementation of Information and
Communication Technologies at the University of Zagreb for ten years. She is a member of EDEN Executive Committee.
Anderson LW , Krathwohl DR (eds) ( 2001 ) A taxonomy for learning, teaching, and assessing: A revision of Bloom's Taxonomy of Educational Objectives, Completeth edn . Longman, New York
Biggs JB , Tang C ( 2007 ) Teaching for quality learning at university: what the student does (3rd ed .). Berkshire, England: Open University Press
Bloom BS ( 1956 ) Taxonomy of educational objectives; the classification of educational goals, 1st edn . Longmans, Green , New York
Bloom BS , Masia BB , Krathwohl DR ( 1964 ) Taxonomy of Educational Objectives (two volumes: The Affective Domain & The Cognitive Domain) . David McKay , New York
Boud D , Falchikov N ( 2006 ) Aligning assessment with long-term learning . Assessment and Evaluation in Higher Education 31 ( 4 ): 399 - 413 , http://dx.doi.org/10.1080/02602930600679050
Brown G , Bull J , Pendlebury M ( 1997 ) Assessing student learning in higher education . Routledge , London
Carless D ( 2007 ) Learning-oriented assessment: Conceptual bases and practical implications . Innovations In Education And Teaching International 44 ( 1 ): 57 - 66
EHEA. ( 2014 ). Bologna Process - European Higher Education Area. Retrieved September 15 , 2015 from http://www.ehea.info/
ENQA ( 2009 ) Standards and Guidelines for Quality Assurance in the European Higher Education Area, 3rd edn . European Association for Quality Assurance in Higher Education , Helsinki, Retrieved September 15, 2015 from http://www.enqa.eu/wp-content/uploads/2013/06/ESG_3edition- 2 .pdf
Falcão R. ( 2013 ). Aloa: a model for aligning learning outcomes and assessment . 45th EUCEN Conference: Transferring knowledge in a globalised world: a ULL responsibility . University of Geneva, Switzerland. 29 -31 May 2013 . Retrieved September 15 , 2015 from https://www.unige.ch/formcont/files/5714/3921/7163/Falcao-CS-FINAL.pdf
Garrison DT , Anderson T ( 2003 ) E-learning in the 21st Century: A Framework for Research and Practice . Routledge Falmer , London & New York
Gil-Jaurena I , Aguado T , Malik B , Cucalón P ( 2015 ) E-assessment in a Master online course. A case study . INTED2015 Proceedings , Madrid, pp 3661 - 3667
Kennedy , D. , Hyland , A. & Ryan , N. ( 2006 ). Writing and Using Learning Outcomes: a Practical Guide . Retrieved September 15 , 2015 from http://www.tcd.ie/teaching-learning/academic-development/assets/pdf/Kennedy_ Writing_ and_Using_Learning_Outcomes .pdf
Lõssenko J. (coord. ) ( 2014 ). WP 3 . 01 -TALOE E-assessment Practices . Retrieved September 15 , 2015 from http://taloe.up. pt/wp-content/uploads/2015/03/AnnexXI_WP3_01_E_Assessment_Practices.pdf
Ministry of Science, Technology and Innovation ( 2005 ) A Framework for Qualifications of the European Higher Education Area . Bologna Working Group on Qualifications Frameworks. Copenhagen, MSTI, Retrieved September 15 , 2015 from http://www.ehea.info/Uploads/Articles/050218_QF_EHEA.pdf
Rodríguez-Gómez G & Ibarra-Saiz MS . ( 2015 ). Assessment as Learning and Empowerment: Towards Sustainable Learning in Higher Education . M. Peris-Ortiz & J.M. Merigó Lindahl (eds .), Sustainable Learning in Higher Education (pp. 1 - 20 ). Springer. http://dx.doi.org/10.1007/978-3- 319 -10804- 9 _ 1
Soeiro A , Martins I & Correia T. ( 2014 ). WP 3.02 ‐ ALOA model with integration of e‐assessment practices . Retrieved September 15 , 2015 from http://taloe.up.pt/wp-content/uploads/2015/03/AnnexXII_WP3_02_ALOA.pdf
Iowa State University ( 2015 ). Revised Bloom's taxonomy. Center for Excelence in Teaching and Learning . Retrieved September 15 , 2015 from http://www.celt.iastate.edu/teaching-resources/effective-practice/revised-bloomstaxonomy/
Vygotsky , L. ( 1978 ). Mind in society: development of higher psychological processes . Harvard University Press.
Wenger E ( 2000 ) Communities of Practice and Social Learning Systems . Organization 7 ( 2 ): 225 - 246 , http://dx.doi.org/10. 1177/135050840072002