Last month I was invited to participate in a workshop organised by the London Knowledge Lab (LKL). Justin Olmanson, postdoctoral fellow of the Institute of Education Sciences and member of Dr. Bill Cope’s research team from the University of Illinois, visited the Institute Of Education (IOE) and the LKL and introduced to us the assessment tool called “Scholar” and its conceptual framework. We then had the opportunity to experience the environment from either a student’s or a teacher’s point of view with hands-on activities.
Scholar is
a ‘cloud computing’, web-based writing and learning environment for learners which brings together formative assessment (diagnosis and feedback) and summative assessment (measuring student progress over time and in comparison with other learners).(Kalantzis & Cope, 2012)[1].
What I liked most:
Its name and analogy
The tool is divided into three modules, namely Community, Creator and Publisher, with two more modules currently under development (Conference and Bookstore).
Students can be part of one or more communities created by the teacher, where they interact with each other, learn to operate and be part of a “scholarly” online community.
The teacher sets the assessment activities and the students start creating content. The interface includes a text editor that allows them to easily edit their text but to also incorporate images, tables, links, videos and audios into their assignments.
scholar | Creator Module
The students not only start interacting as scholars but also engage in digital scholarship, familiarise themselves with many different formats and hence the tool prompts the development of skills in a variety of literacies, such as Media, Digital and Information Literacies.
Scholar’s Conceptual Framework
The feedback area promotes peer-assessment; the students start reviewing the assignments of their peers, add annotations in areas of interest and contribute their reviews against the set assessment criteria.
Overall the students are engaged in a holistic scholarly environment, where they are content creators, part of a community, peer-reviewers and publishers of their final assignments.
In hindsight, as students, most of us graduated with minimal knowledge of and without any experience of the scholarly communication cycle. Even now many new Professionals graduate with many questions around the academic publishing and communication. I feel that the tool nicely embarks students in the aforementioned cycle, making them feel as scholars.
Supporting deep learning
The literature around the diagnostic nature of assessment is vast and continues to generate a vivid academic dialogue especially with the affordances of new learning technologies.
Scholar’s scaffolding allows the assessment activities to become learning activities. The students are able to go back to the revisions of their work, reflect upon their peers’ annotations and track the development of their thoughts through the course of the assignment from the first draft to the final version.
Source: Deep and Surface Approaches to Learning http://exchange.ac.uk/learning-and-teaching-theory-guide/deep-and-surface-approaches-learning.html
Scholar encourages:
- interaction with both multiple media and peers and facilitates dialogue;
- the formation of groups so that advanced students are able to mentor the ones in earlier stages of study, should the teacher consider it suitable;
Find the printable cards at http://jiscdesignstudio.pbworks.com/w/file/59487220/assessment_feedback_printable_cards.pdf
Scholar:
- allows learning to become social and enables the creation of knowledge within communities;
- enables the development of a blend of Academic and employability skills, such as team-working, analytical thinking, narrative and argumentation skills and digital scholarship;
Find the printable cards at http://jiscdesignstudio.pbworks.com/w/file/59487220/assessment_feedback_printable_cards.pdf
Scholar:
- allows learners to peer and self assess their work against set criteria;
- supports differentiated learning as opposed to the “one size fits all” approach by giving opportunities for a variety of interactions;
- actively involves students in the assessment process rather than their being passive recipients of a grade;
Find the printable cards at http://jiscdesignstudio.pbworks.com/w/file/59487220/assessment_feedback_printable_cards.pdf
- Students’ reviews and annotations reveal threshold concepts, or areas that need attention during the in-class instruction;
- Student responses can provide insight into the classroom dynamics;
- it enables the creation and use of quizzes and surveys that teachers can use to seek feedback, or adjust a similar to Stephen Brookfield’s Critical Incident Questionnaire (CIQ) to be able to iterate their teaching;
- Quizzes can also be used for Ipsative assessment prior to any instruction and hence teachers can create adaptive learning activities.
From Educators to Educators
The tool is the outcome of the collective work of a research team supported by the U.S. Department of Education Institute of Education Sciences across four grants. Unlike the majority of technologies that have been developed with the commercial domain’s needs in mind, and were later embraced by educators who tried to find innovative ways to use them in the classroom setting, Scholar is one of the very few learning technologies that have been created by educators to support the learning process with diagnostic assessment being in its core.
How it would work as part of an Information Literacy Assessment plan
Megan Oakleaf (2009) had pointed early on that Librarians can demonstrate impact in their Institutions’ learning and teaching missions through the deployment of Information Literacy assessment plans. Scholar can facilitate this ambition and become part of the proposed assessment plan.
Even when Librarians cannot be physically embedded within departments, the tool can ease the way towards meaningful collaboration between Academics and Librarians and therefore create a fertile ground for innovation.
Andrew Walsh (2009), in another study reviewing the relevant literature up to 2007 to identify reliability and validity of Information Literacy assessment methods and tools, suggested that during the assessment design it is important to balance our needs with tools that are easy to administer and at the same time can assess what our Information Literacy-related outcomes describe. From my short experience of Scholar I can see that it provides many opportunities for librarians to administer assessment and export qualitative data that prove whether learning outcomes have been achieved.
Alignment with the Higher Education Academy’s Professional Framework
Assessment affects all the aspects of our practice and the quality-enhanced approaches we take to teaching and students’ learning.
Scholar’s theoretical architecture is underpinned by teaching, learning and assessment theories (Knowledge) and seamlessly supports the core teaching activities and professional values.
The UK Professional Standards Framework (UKPSF) available at http://www.heacademy.ac.uk/assets/documents/ukpsf/ukpsf.pdf
Some thoughts on Improvement:
Seamless integration in MOODLE
It would be great if Scholar integrated or were able to communicate with MOODLE. Students are tired of logging in many different platforms, having to remember numerous sets of credentials to engage with isolated Institutional services. An effortless communication between the Virtual Learning Environment (VLE) and an assessment tool would help students make conceptual links between course content, activities and assessment. Moreover, it would be easier for platforms to exchange and combine data sources from different areas of the VLE.
Learning Analytics will grow in interest and platforms will need to be able to bridge departmental silos and merge data from many different sources in order to give reliable studies on students’ achievement or under-performance.
Clearer dashboard visualisation of complex data
Scholar can generate a variety of data that report on students’ performance. The visualisation of such a complex data sets is currently following a linear graphical representation. I would personally like to see a less complex data visualisation which is represented in a cycle rather than in an overwhelming number of bars. Colour annotation of different attributes and values would make it attractive to the eye and easy to follow. A nice example of such a prototype is the visualisation of the learning designer tool which visualises the teacher’s preparation time. [2]
In conclusion:
It was a great pleasure to have been invited to participate in this small hands-on workshop and to have met with educational researchers. Assessment for Learning (AfL) needs to be one of the main considerations when designing Information Literacy courses, curricula or just stand-alone sessions. I can see a great potential for those who are able to use Scholar in their own settings for their students’ learning.
Ramsden (1992) advocated that the assessment procedures have a major impact in students’ learning and in the way they engage with learning as a whole. Scholar can give educators a variety of both formative and summative assessment options that fit the kind of knowledge and skills needed in the 21st century.
References:
Biggs, J. (1999). Teaching for Quality Learning at University, SHRE and Open University Press.
Entwistle , N. (1988). Styles of Learning and Teaching, David Fulton.
Higher Education Academy. (2011). The UK Professional Standards Framework for teaching and supporting learning in higher education. Available online at http://www.heacademy.ac.uk/assets/documents/ukpsf/ukpsf.pdf. [Accessed 11 June 2013]
Oakleaf, Megan (2009). “Writing Information Literacy Assessment Plans: A guide to best practice. Communications in Information Literacy no. 3 (2):80-89.
Ramsden, P. (1992). Learning to Teach in Higher Education, Routledge.
Scholar website available at http://learning.cgscholar.com
Walsh, Andrew (2009) Information Literacy Assessment: Where do we start? Journal of Librarianship and Information Science, 41 (1), pp. 19-28 (Open access version available at http://eprints.hud.ac.uk/2882/)
Notes:
Scholar images were used here with kind permission of Dr. Bill Cope.
Find more about the printable card and the Viewpoints JISC project outputs at http://jiscdesignstudio.pbworks.com/w/page/29227748/Viewpoints%20project
Eleni's First Steps 
Pingback: Assess and be assessed as a Scholar | Informati...
Pingback: Assess and be assessed as a Scholar | Assessmen...