1This discussion focuses on a project from 2002-3 entitled ‘Visual technologies and their assessment in undergraduate teaching and learning’.1 It was funded by C-SAP, the Higher Education Academy Subject Network for Sociology, Anthropology, Politics.2 The object of C-SAP network was to support teaching and learning within anthropology and cognate subject areas, and to improve the student learning experience in Britain until December 2011, when its funding was due to end. The discussion refers to activities from over ten years ago, but the fact that a conference session on visual assessment was taking place in in 2013 is evidence of the relevance of this historical project to contemporary visual anthropology teaching, in theory and practice. Because of developments since 2003 there are new challenges which I’ll refer to in my closing comments, where I also consider what evidence the project contributes to debates about whether should we define ethnographic film and its anthropological relevance.
2The project was designed in the context of a regular Anthropology undergraduate honours course, and not a special Visual Anthropology programme. It aimed to develop pedagogic models that diversified methods of teaching, learning and assessment away from standard lecture/essay type formats, and to provide clear guidelines for their assessment criteria. The project aimed to develop the use of visual media in two undergraduate modules, the History of Anthropological Theory and Visual Anthropology. It involved training students in visual methods so that they could and represent their findings using words and images and complete visual ethnographies in different ways. Time was set aside to discuss and determine assessment criteria with the participating students in both these modules. The resulting innovations were also intended to enhance the students’ learning experience and problem-solving skills, as well as giving them desirable I.T. skills for the job market. The project also developed recommendations about curriculum design and its place in the overall structure of the degree scheme, including how to allocate contact hours to practical work. We intended the findings about assessment to be helpful for other lecturers in anthropology, and produced a web site which included samples of work and project documents to disseminate the results.
3A comment about my background is pertinent here. My active involvement in ethnographic film dates back to 1986 when I was as Leverhulme Film Fellow in documentary at the National Film and Television School (NFTS, Beaconsfield, UK). This scheme was a collaboration between the NFTS, the Royal Anthropological Institute of Great Britain and Northern Ireland (RAI), and the Leverhulme Foundation who funded the project for three years. The object was to train anthropologists to make ethnographic films; each participant was to make a film about his or her research, working with another NFTS documentary student. It turned out that three other anthropologists were trained on this scheme: John Bailey and Paul Henley, who held fellowships for two years, and Marcus Banks who was a fellow for one year at the same time as me. This training inspired my subsequent activities which included teaching of Visual Anthropology in the then Department of Sociology and Anthropology at Swansea University, as a third year undergraduate special option, and as part of Research Council recognised postgraduate research training in Anthropology, and later Geography. I also acted as external examiner for Visual Anthropology Masters degrees in Manchester University and Goldsmiths' College, London University, and served as reviews editor for the journal Visual Anthropology for 12 years. From 1988 when I completed my NFTS training I became a member of the Film Committee of the Royal Anthropological Institute of Great Britain and Northern Ireland (RAI), and in that capacity acted as a selector at most of the RAI International Ethnographic Film Festivals and as judge at two festivals; I was also was a selector twice for the Göttingen International Ethnographic Film Festival. My latest involvement in the academic institutionalisation of ethnographic film is as a member of the editorial board of this journal, AnthroVision. This short biography is offered as evidence for my interest in ethnographic film, but the project in question also dealt with other forms of non-verbal representation, as I now explain.
4In the History of Anthropological Theory module, second and third year undergraduates produced their research projects for final assessment in the form of CD-ROMs.3 Previously, students had been assessed by a portfolio of project work for assessment about either an early anthropologist -- traveller, explorer or theorist -- or an expedition. Producing this portfolio was problematic for the students because of the costs of photocopying sources in duplicate for final submission according to departmental regulations. I suggested to Dr Margaret Kenna, my colleague who taught this module, that these problems could be overcome if the students presented their work on hypermedia instead of paper-based portfolios. She agreed, and we decided to ask the students to submit their projects as CD-ROMs, in the first instance.
5From contemporary perspectives, certain aspects of the ‘General Training Documents’ on the project website will seem bizarre. Back in 2003, scanning documents and making CD-ROMs was a major technological challenge for the students. They needed to be taught how to do this. Part of this was done through a 50-minute ‘scanning training’ session. This was deemed equivalent to a 1,000 word piece of ‘formative coursework’ for the module, to prevent the students from feeling burdened by the extra contact hours needed for this training. The training session also introduced them to a new way of presenting information, called Power Point. These technologies were so new and unfamiliar that the project also needed the trainer to hold weekly ‘Surgery Hours’ and also ‘Emergency Surgeries’ in the week before the work was submitted, to reduce student anxiety about the technical demands of their individual projects. It is striking what we take for granted now in terms of technology – and also striking how little we exploit it, a point I return to later.
6At the start of the 21st century, there were very limited resources on how to assess non-verbal student work. C-SAP’s booklet on ‘portfolios’ from the Generic Learning Centre (Baume 2001) had already been helpful in devising the hard-copy portfolio previously used to assess The History of Anthropological Theory, but we were unable to find any resource for using the CD-ROM as a learning device or for ways of assessing it. We developed our own criteria for the assessment of the projects on CD-ROM, as follows:4
71. Content (relevance of choice of topic; usefulness to another student of sources listed or sampled, or websites to which hyperlinks connected; any evaluation of relevance of these sources)
82. Structure and Presentation (categories/ labels under which information collected; ease of ‘navigation’; arrangement and selection of material; use of maps, diagrams, illustrations)
93. Written Expression (i.e. student's own words, as commentary, if used)
104. Evidence of Technical Competence (evidence of basic competence in scanning, creating hyperlinks, cutting and pasting)
115. Overall Assessment (Suggested balance: Content 35%; Structure 35%; Written Expression 10%; Technical Competence 20%) (CD-ROM Assessment form,
12The suggested balance of the Overall Assessment was the result of consultation with the students. They considered content and structure to be the two most important criteria of evaluation, followed by technical competence. They thought that written expression should be given a low priority, hence the 10% allocated to it in the overall assessment.5
13The fifteen second year and twenty four third year students all demonstrated a basic technical competence in scanning. A number displayed very high levels of technical proficiency in design and presentation. A sample of work was marked by both project participants. The highest marks were gained by those students who had devised labelled ‘pathways’ through the material and evaluated its usefulness for other users (the Content and Structure criteria). As well as technical competence in scanning, and in some cases the use of Power Point, the students had also acquired valuable skills in structuring the presentation of such materials in a logical and ‘user-friendly’ way. This was demonstrated by an unintended consequence of a student’s original research into Edward Tuite-Dalton, a distant relative from Victorian times who had published an ethnographic survey of a part of India, using family documents and other memorabilia. A library in Canada with an interest in Tuite-Dalton found her project online and contacted her about her research, enabling them to supplement their materials. Other projects included explorers such as Lady Mary Wortley Montagu, Isabella Bird, and Mary Kingsley, expeditions such as Torres Straits, Jesup North Pacific, Lewis and Clark, and one student who was also taking Visual Anthropology worked on Robert Flaherty.
14At the end of the module, one CD-ROM was handed in without a student number to identify its author. This mystery submission turned out to be a humorous comment on the module by the students who had worked together collectively to express their frustrations and difficulties in learning new technical skills in an entertaining manner. It provided a different form of feedback on the learning experience of the participants, and also showed how much they had succeeded in conquering their fears of new technology and ways of learning, and how they were able to play with the technology as a team.
15I now turn to the Visual Anthropology module which I convened and taught with additional tutorial and technical support. This module was taught to third year undergraduates as a ‘practical’ 55 hour course that was twice as long as normal 20 credit modules in the department. In previous years students on the module had made films, but only their reports and essays had been assessed. As a result of the project, final assessment comprised a 10 minute video film and a report for 50% of the mark, and for a theoretical essay for the other 50%. Students had an option to produce a photo essay instead of a film, but no one chose to do this. The report took the form of a 1000-word discussion about the making of the project. Although the students worked in teams, each student had to write his or her own report. Students were required to discuss how they had decided about shooting (how and what); editing (selection, construction of narrative through the ordering of shots, etc.); the nature of the contract between the student(s) and the subject(s) and any other relevant ethical issues; and finally, how they worked as a team.6
16The teaching of the module was based on lectures and practical training in camera, sound and editing, using a range of exercises and feedback sessions. The approach was based on my own documentary training at the National Film and Television School in the 1980s which was influenced by observational cinema and participant-observational cinema represented in the work of David and Judith MacDougall and many others. For the final projects, students had access to three cameras over three weeks. The students (with one exception) worked in teams of two, to research and produce ‘portraits’ of persons or places in and around Swansea. They filmed 60 minutes of 'rushes' which they then cut into films no longer than 10 minutes. This process was very compressed; there was no time for feedback on rough cuts. All guidance was from the tutor, and the lecturer refrained from seeing the work until it was finished in order to be able to assess the film impartially.
17The videos covered a range of topics chosen by the students, including a family sportswear business, a circus-skills training session, school visits to the University’s Egypt Centre (a reflection of a joint honours Anthropology and Egyptology student doing the module), an art class, skateboarding, a woman working for the cycling campaign group SUSTANS, and anti-war demonstrations. One example on the website is ‘The Need to Swim’ (http://www.swansea.ac.uk/visualanthropology/movie01.html). The subject is a swimmer who used to swim everyday in Langland Bay, near Swansea, to help alleviate the arthritis in his knees. This film appears very simple, but was very carefully thought out and structured to show the swim as a special ritual, in which the swimmer is separated from daily life and undergoes a transformation. Anthropologically it represents visually Victor Turner’s ‘ritual process’ through the images of the swimmer’s transformation into a sea creature and his return to dry land and his daily life, with this arthritis temporarily relieved by his swim. The clip on the website shows developing shots which follow the action, as the swimmer changes his dry land watch for a waterproof one and becomes shadowy creature of the sea; the transformational effect of filming him against the light as he enters the water was a deliberate choice, not an accident.
18Criteria for assessing video work were derived from the Society for Visual Anthropology guidelines, and from assessment criteria for dissertation films submitted to university courses such as the MA in Visual Anthropology, Granada Centre, Manchester University, as formulated in 2002, and guide lines provided for the selection of ethnographic films at festivals. The criteria for assessment for the video and report which together formed 50% of the course mark were represented on the marking sheet as follows7:
191. Filming/ Production
20a) Evidence of competence in basic production techniques. (quality of camera and sound work, effectiveness of questions if used)
Shooting: is it steady? Are the camera movements well-motivated, following the action?
Use of available light: can we see what we need to see? Is there excessive glare or darkness?
Are sound recordings audible and well-balanced?
If questions have been asked, do they fit in or do they intrude?
21b) Evidence of adequate research, planning and content selection (range of situations, planning, relationship between team and subject(s), ethics, inspires confidence in what we see)
Is the range of situations filmed sufficient to explore the topic, given the time limits imposed by the module?
Can we see signs of sufficient advance planning?
Is there evidence of a good working relationship between the team and the subject(s)?
Does the content satisfy any ethical concerns we might have?
Does what we see inspire confidence in the filmmakers’ representation of the subject?
222. Editing/ Postproduction
23a) Evidence of technical competence in basic editing (construction of narrative, effectiveness of cuts, order of scenes, of pace and mood, non-observational techniques, if used)
Do the selected shots contribute to an overall ‘story’?
Are the cuts from one shot to the next satisfactory?
Is the ordering of scenes effective?
Is the pace and the mood appropriate to the subject?
If used, are non-observational techniques, such as voice-overs and other forms of non-synch sound justified?
24b) Evidence of interpretation and judgement in producing a coherent account of the topic (clarity of topic (see also billing), narrative development, (anthropological relevance or implications))
Is the topic of the film clearly established in relation to the billing in the report?
Is there a sense of narrative development (beginning, middle, end)?
(optional: to what extent does the film engage with or raise an anthropological issue?)
253. Evidence of learning and reflexive realization as demonstrated in the report (billing, quality, fairness and accuracy of self-evaluation) 8
How well does the report trace the process of decision-making in planning?
How well does the report describe the reasons for changes of approach/topic/focus during filming?
How clear is the explanation of the editing process (selection, construction of narrative through the ordering of shots, etc.)?
How fairly does the report evaluate the effectiveness of the teamwork throughout the whole process?
How fair and sensitive is the discussion of the nature of the contract between the student team and the subject(s) and any other relevant ethical issues?
To what extent does the report demonstrate an accurate self-evaluation of the strengths and weaknesses of the film produced?
Is there any attempt to relate the film to film in general and anthropology?
26In contrast to the assessment criteria for the CD-ROMs, there was no suggested percentage breakdown between the three elements in the overall mark. There was a reason for this. In discussion about how to assess the films and reports, the students were worried about being assessed for technical competence because this was the first time that any of them had made a complete video other than in the training exercises. It was agreed that the assessment guidelines applied to what was being aspired to, rather than necessarily being achieved to perfection. The report should guide the marker’s evaluation of how well each student has learnt from the mistakes which will to various degrees be evident in the film. For this reason, although in practice it was impossible to split the marks precisely because of the synergy between the report and the video, it was agreed that the balance of marks would be 50-50 between the film and the report, so that the evidence of errors could be redressed by evidence of a student’s recognition of these. The students also proposed that each report should open with a short summary or ‘billing’ of the film. And finally, to help the markers appreciate the extent of the challenge and the degree of flexibility experienced by the teams during filming, it was agreed to include the preliminary proposal as an appendix to contextualise the learning process and outcomes, but not for this to be assessed. After a long discussion, the students also decided that the assessment of anthropological relevance should only apply to the written report, and appear as an optional criterion in the assessment of editing and postproduction.9 Their struggle with this issue reflects longstanding debates about how anthropological ethnographic film should be, an issue that will be considered in my concluding remarks.
27The actual process of assessment was complex. The course convenor read the reports. There were two one-hour seminars run as focus groups for assessment where we all watched the films, including the editing tutor. The second marker (who followed a similar procedure alone) and the external were asked to view all eight films. All films were assessed in the class of 2.1 (60% and above) and upwards, but some of the reports did not show such a high level of attention, so some marks were lowered as a result. Ironically, everyone would have gained higher marks if my original proposal of 65% for the film and 35% for the report had been followed.
28There had been steep learning curves in both modules in the project, but the students were generally enthusiastic about the opportunities to use these new methods as part of an academic curriculum. Their comments about skills acquisition and their progress in knowledge, thinking and theory, research and reading in the Visual Anthropology module were evidence that the practical work had developed their understanding further than the more classical forms of assessment would have done10:
‘Develop arguments on the considerations anthropologists take an account of, in documenting social situations. Filming techniques.’
‘Experience of making a film, much influence on how I read about visual anthropology’
‘On the importance of producing a visual work. Learnt to be tolerant in relation to problem do with the team partner. Became aware of the problems an anthropologist might have in trying to produce visual ethnography. Importance in producing film that represents reality.
29The markers were also pleased with the results. As the-then external examiner, Dr Garry Marvin wrote:
‘Two areas that are new to me this year deserve the highest praise - the production of CD ROMs for the History of Anthropological Theory and the video films for Visual Anthropology. In both of these courses the students have been encouraged to use their imagination and skills to produce truly original work rather than repeating and commenting on the work of other anthropologists… The ability to design a project on CD ROM with all the necessary research into web resources, the creative use of graphics and the hyperlinks etc and the skills of visual production in terms of design, filming and editing are skills that will certainly have a potential use outside the academic world. These two courses successfully bring new media into productive contact with academic anthropology and constitute a most valuable addition to the range of anthropology on offer at Swansea…’ (Gary Marvin cited pp 3-4, Final Report, http://www.swansea.ac.uk/visualanthropology/framework.html)
30An unintended consequence of the video work in this course was that three students were able to participate in a field school in California organised out of the University of Glamorgan by Dr Teri Brewer. I acted as Assistant Director of the School and Director of the Visual Project which aimed to produce a 25 minute video. I provided a day’s training day, and my three ‘Visual Anthropology’ students from Swansea acted as tutors to the 15 other students from Swansea and Glamorgan. They each lead a team equipped with digital camera and sound, and each team had its own daily agenda. The 10 hours of rushes were edited by Jen Hughes, one of the Swansea Visual Anthropology students, into the first version of the film, A Matter of Interpretation. This was subsequently re-cut into a shorter and stylistically different version by an editor working for the field school director (https://vimeo.com/127845326). It was a function of the Visual Anthropology module's effectiveness that it had equipped students to be able to train others – this was not a skill that I would have anticipated that they would acquire so quickly and effectively. This alone was proof that the Visual Anthropology module had provided solid filming skills that could be transferred to a training context. For he Swansea students, the field school was extra-curricular and did not contribute to assessment for their final degree. The participating Glamorgan students were assessed by their written work only.
31Visual assessment cannot be essentialized any more than can ‘written assessment’. Assessment criteria have to be associated with the overall learning outcomes and skill acquisition for the module, as well as reflecting disciplinary benchmarking criteria and standards. To discuss assessment we have to know what we are assessing. So why is the assessment of film and multimedia important? For our purposes, I would argue that it is because of the relationship between anthropology and visual practices.
32In the two modules discussed in this account very clear rules and guidelines were provided to the students. Visual Anthropology taught them to film and edit following the conventions of observational cinema. They had to follow the action, shoot developing shots with a minimal use of zoom, and edit as closely as possible following the order of events in the rushes, using plain cuts, not fade and wipes. The assessment criteria reflected how well the students had understood and worked with these rules.
33Apart from the formal bureaucratic assessment criteria, ethnographic film is so important and relevant to training in anthropology because it provides raining in attitudes and embodied presence in research. The process of making a film enacts, dramatizes even, and reveals the anthropological research relationship with the subjects. Such relationships reflect an investment of time in the situation being documented. Even if this is not a long-term personal relationship with everyone in the film before shooting begins, it is a less narrowly opportunistic or exploitative relationship than those found in short sharp shoots for reportage documentaries. Personal accountability, moral, ethical relationships, and a commitment to a broader context of increasing understanding and diversity is surely a mark of anthropological relevance within ethnographic filmmaking.
34Since the project was completed, increasing attention has been paid to the visual and to the other senses. For instance, in his book of the same title, MacDougall uses the concept of the ‘corporeal image’ (2006: 127) to argue for film as another way of examining how our embodied seeing and being produces our understanding of the world; ‘…[S]cholarship should be an expression of one’s sensibilities as well as one’s mind but this ideal synthesis is not easily achieved (2006: 123). Despite the strength of MacDougall arguments, in anthropology the senses have tended to remain more important as the object of analysis than as a method, and film continues to be viewed with suspicion. In the humanities more widely, this is less the case. Having spent years being told that my films would not count for much in research assessment exercises, and being unable to find funding for projects that included films, research councils in the UK such as the Arts and Humanities Research Council offer funding in ‘practice-based research’ which includes filmmaking. This is an excellent development, but sadly one that occurred not long before the drastic reduction in the government’s funding for research. Given the broader recognition of practice-based research, it is more urgent than before to bring this kind of activity into academe curricula, and for undergraduates, not just postgraduates.
35This is by no means straightforward. Practical difficulties deter lecturers from using diverse forms of assessment in a general undergraduate anthropology degree scheme. During the C-SAP project at Swansea, we were faced with problems of finding technical support and the funding to pay for it. As the university set up more computer labs, the editing and CD-ROM training became much easier, and more students could enrol on the Visual Anthropology option. However, as technological support has improved, increasingly unimaginative managerial thinking about teaching and learning is contributing to greater conformity in courses in the UK. University lecturers have to think carefully how to balance their time between teaching, research and administration. Indeed my Head of Department asked me to decide if I wished to continue put so many hours into the Visual Anthropology module at the expense of time that could be spent on my research. This was a decision I never had to make. In 2004 the management at Swansea University decided to withdraw to support the Sociology and Anthropology department in a process termed 'disinvestment'. The Visual Anthropology module was taught for the last time in 2006, and the last Anthropology students graduated from Swansea in 2007. This disastrous sequence of events is another reason for the historical nature of this account, and why I wish to put on record the project, because its anticipated dissemination was cut short very abruptly.
36There are implications from my somewhat 'retro' account for contemporary contexts of understanding and communicability and the future assessment of film-based student work. A criterion for the CD-ROM in the History of Anthropological Theory was to structure the presentation of materials in a logical and ‘user-friendly’ way; accessibility was important. This was not included in the assessment criteria for the videos, and it probably should have been. The students making the films were uncertain how ‘anthropological’ their work was and how to communicate that aspect of their work. How we teach our students to communicate ‘anthropologically’ in their films, and how explicitly, is an ongoing challenge for both Visual and General Anthropology.
37It goes without saying that since the project was completed in 2003 there has been a radical change in access to images of performance on YouTube and Vimeo. But the ethnographic context of these images is rarely clear, and such images tend to produce fragmentary impressions and endorse prejudice, rather than anthropologically-founded understanding. Multimedia now offers many kinds of experimental ethnographic explorations. These are innovative and exciting approaches to issues of analysis and representation in anthropology, but they require great care to avoid excessive confusion about context and legitimacy. The navigational structures that students needed to learn for their CD-ROM project are now becoming more natural to digital 'native' than linear structures, but it is still important for students to learn how to produce the narrative structures of linear film within their video projects., to counter increasing fragmentation and brevity in how people see moving images, exemplified by YouTube. It is also important for students to engage with documentary film, and to do more than a single location shoot. When there is only one location (as in the video project, The Need to Swim), narrative development is still important, whether as a story or a more formalist representation of an idea. A short film can achieve development but is more difficult to achieve than in a longer film, so it needs an even stronger concept and visual imagination. This also produces the ability to learn and understand in the epistemological style that we call 'anthropological'.
38Credibility in documentary film in general has been hard earned, but it is at risk. Current experiments in ‘faction’, where ‘fact’ and ‘fiction’ are combined, risk undermining a hard earned believability, albeit within the context of a naturalistic aesthetic which conceals its mix of artifice and actuality. The use of re-enactments or films-within-films, is a case in point. A good example is Tobias Wendl’s film Ghanaian Video Tales, which makes use of different kinds of frame to make clear which sections of his film are about Ghanaian filmmakers and which are their own films. This helps to preserve the credibility of what the audience is seeing. These issues, of courses, are founded on complex philosophical questions which are central to general anthropology, not just visual anthropology. How to counter this trend is a challenge to visual anthropologists, and to our assumptions about realism and naturalism.
39A final point relates to increasing inter-disciplinarity in the digital era. In British universities, ‘studies’ are replacing many 20th century academic categories and structures for intellectual enquiry - or 'teaching and learning' in university-speak. This has major implications for the relationship of Visual Anthropology to Visual Sociology, not to mention to Media Studies, Communication Studies and Film Studies. A question we need to ask is this: does inter-disciplinarity make it necessary to be clearer about what distinguishes ethnographic film from documentary? Many of us digital immigrants have resisted defining ethnographic film and have favoured innovations from other domains of visual practice. Should we follow Ruby’s (2000) distinction between ethnographic and anthropological film? Should we attempt to specify criteria for anthropological relevance in ethnographic filmmaking? Do we still value a generic boundary between ethnographic film and documentary? Or has the connection between ethnographic film and anthropological research been broken? (The award of the Basil Wright prize to the documentary The Act of Killing at the 2013 RAI International Ethnographic Film Festival suggests that is has.) If this is the case, is ethnographic film a meaningful category?
40Most Visual Anthropologists have tried to avoid defining the elements we know make a film ethnographic. However, as disciplines lose their boundaries, and narrative is challenged by non-linear and non-realistic forms, we urgently need to situate the position from which we define and assess ethnographic film. Despite my own interest in new media I personally would advocate an anthropological position to do this, even as anthropology is undermined in so many ways. Perhaps it is through the struggle to answer these questions within Visual anthropology that we will contribute to the orientation of Anthropology’s general relevance for the future.