The Credibility of Electronic Publishing

 

Introduction and Overview

Raymond Siemens


Introduction: Scholarly Publishing at its Source, and at Present

One of the most compelling arguments to assist us in the consideration of the validity of various types of newly-emerging electronic scholarly publications, what may be called the argument ad fontes, holds a place also as one of the most implicitly revealing arguments for its slow acceptance -- especially when one considers how publication relates to notions of credibility in today's professional academic environment.

The Argument Ad Fontes

In general circulation for some time, the argument ad fontes suggests that we consider the source of our formalized academic publication process; it operates, in essence, as follows.  In earlier times, when formal methods of inquiry and dissemination as we now know them were being shaped, the exchange of ideas and knowledge and the advancement of scholarship was facilitated as much by private exchanges and the circulation of private manuscripts and correspondence as by other means.

The establishment of the first scientific journals, in Britain and France during the seventeenth century, saw the beginnings of academic publication's formalized dissemination, concurrent with the development of the role of scholarly editor -- the scholarly editor, in this case, being someone who took on the task of re-circulating materials of interest to a community of scholarly readers.  In addition to being responsible for disseminating those materials, the editor also helped determine which materials were of value enough to warrant dissemination to that community.  While editorial boards and methods of review (methods that act, to this day, as additional assurances of value) would soon rise out of this, much of value still happened outside the more formalized processes of exchange that were emerging, and much of value would continue to be exchanged in forums outside of those vetted by editor and editorial board; and so it does to this day.

Even so, and even in the earliest stages of scholarly publication's development, not all forums were afforded the same level of attention -- nor was there, even early on, the possibility of ensuring that all valuable forums received the same level of attention.  Exchanges of ideas through conversation might benefit chiefly those who had the fortune of being present at the time of the conversation, but conversations of particular utility and interest could be reported in correspondence, which in turn could be redistributed to a larger group via print, if resources and inclination existed for this to happen.  The print record (journal note or article, monograph, &c.) could then be preserved, again providing that the community served by the print record had the means and inclination to do so.  Ownership of ideas and materials was at issue variously; but, on the whole, emphasis was on ensuring that valuable ideas circulated and became part of growing, documented, bodies of knowledge.

The argument ad fontes is often used to suggest that we might turn to earlier models of scholarly exchange -- not only the emerging formal methods that we now accept, but also those that were and are treated less formally -- and consider their possible relationship, even if only metaphorically, to what we now refer to as "new" types of scholarly exchange that are made possible by the electronic medium, specifically via the computer and the internet.  Moreover, by highlighting the validity of contributions disseminated via and made possible through such models of exchange, this argument is used to suggest that academics should be willing to consider the valuable ideas and information that can be made available to us through new systems of interaction and dissemination.

The Analogy of the Source, and Conditions at Present

The argument ad fontes is not ineffective and, indeed, all the available evidence suggests that academics today are very willing to consider how the electronic medium can facilitate dissemination of work and collegial interaction -- an active consideration that is underway in areas well beyond those where computing and networking technology are unavoidably at the heart of their pursuits and concerns.  Among the group of teachers and scholars that makes up the Social Sciences and Humanities in Canada, for example, most make use in some way of electronically-based academic resources and electronically-facilitated exchanges in their work.

Clearly, recognizing the benefits provided by the new medium for the work in the Social Sciences and the Humanities is not an issue.  As in past, academics are today motivated by the same desire to work within communities of scholars and academics that share our interests and, together, to offer our contributions freely for the advancement of knowledge.

Even so, there is a reluctance -- among the same group that makes use of electronically-based academic resources and participates in electronically-facilitated exchanges --  to publish in refereed electronic outlets or make their scholarship available electronically in other ways.  Anecdotal evidence regarding this abounds, but figures help us quantify this more accurately: only approximately one fifth have published at all in refereed electronic outlets, and just over half have made at least some of their scholarship available electronically in some manner.

While accurately reflecting patterns of academic interaction that hold true to this day, the argument ad fontes, as it is typically employed, does not assist us directly in accounting for this apparent reluctance.  As it is typically employed, though, the argument ad fontes favours discussion of emerging academic publication practice over discussion of what we might best refer to as the "academic culture" of that time, especially that compared with the same of our own time.  While the desires and goals of academic work are translatable from past times to our own (like various types of scholarly interaction are also suggested to be), the environment in which academic inquiry is, today, largely conducted has undergone considerable change over the last several hundred years.  As such, it is important for us to remember that we have moved away from the age of the learned and inquisitive amateur that saw the foundation laid for today's formal academic publishing; and, indeed, we have moved toward the age of the contemporary academic professional, the many facets of whose professional life may necessitate the embrace of well-entrenched print-based academic publication methods, ownership practices, and archival methods -- even if that professional's primary motivation is founded in the desire to work within a dynamic community of scholars and academics that share their interests and offer their contributions freely for the advancement of knowledge, and even if electronically-facilitated methods of dissemination and interaction would greatly enhance their ability to do so.

In short, the academic climate is, at present, one in which pragmatic, professional concerns appear to act as barriers to the acceptance and use of venues in the electronic medium that might, quite clearly, better facilitate work towards goals common to all academic and scholarly activities.

Drawing from the Source: General Suggestions

By highlighting the validity of contributions to knowledge disseminated via and made possible through innovative models of exchange that fall outside of those that see professional-acceptance today (as well as within), the argument ad fontes does helpfully suggest that we should be willing to consider ways in which these models of exchange can best assist us in our professional work.  In doing so, it offers much to our overall conceptualization of ways in which new forms of academic dissemination and interaction are made possible by the electronic medium -- itself  a very valuable activity.

Beyond suggesting that there is good reason to be open minded about the validity of evolving forms of dissemination and interaction, the argument ad fontes also assists us as we consider, perhaps, exploring some key differences in academic culture between earlier times and our own.  Such an exploration may not, itself, help us pragmatically and directly address matters associated with the professional concerns of publishing academics and scholars, but it does draw attention to pertinent concerns and can assist us -- again, overall -- when we consider how electronic publication relates to notions of credibility in today's professional academic environment.

On one level, this report attempts to function similarly, making suggestions about overall patterns of professional interaction amongst those the report hopes to serve, noting how aspects of such interaction relate to the concerns specifically noted as being related to matters of electronic publishing's credibility.  On another level, it hopes to focus attention clearly upon a number important tasks directly related to the pragmatic acceptance of electronic publishing's credibility in Canada (as elsewhere) at the moment.  Stated generally, these are:

  1. working with the larger academic community, to emphasize that electronic publishing -- perhaps even a broadly-construed view of what publication is in the electronic medium (i.e. those forms suggested by the argument ad fontes and their contemporary, electronically-facilitated counterparts) -- needs to receive proper and due consideration as a valuable professional activity;
  2. working with those who are responsible for the review, publication and preservation of academic materials in electronic form, to ensure that the best qualitative standards, reliable dissemination and ongoing availability and archiving are wholly integrated into their publication and preservation processes;
  3. ensuring that all involved groups work together toward what is, indeed, the common end of the advancement of knowledge;
  4. and, as an essential component of all the above, taking on the duty of raising awareness not only of accepted resources that are available to facilitate scholarly interaction, but also acknowledging professionally-approved venues for such interaction and making available clear criteria for assessing such venues.

In a number of ways, and with more precision than demonstrated above, the individual sections making up this report draw attention to such tasks.  We present this document in the hope that it will be of service to the Humanities and Social Sciences Federation of Canada, its constituent organizations and related agencies, and beyond.

[Note: Aspects of the argument ad fontes, and its criticism, can be found in many places, among them the exchange between Harnad, Fuller, and others in the Times Higher Education Supplement (May 12, 1995); historical, and other, aspects of this argument can be found in, among others, Fjällbrant, and Schauder.


 

Academic Credibility and Electronic Publishing

There is, perhaps, some irony to the fact that the argument ad fontes suggests, initially and explicitly, why academics should embrace the electronic medium for its facilitation of the sorts of dissemination and interaction that drive the Humanities and Social Sciences and, in the implicit observations it encourages about "professional" academic culture between past times and today, further suggests why electronic publication is still, in a number of circles today, an act that is not yet understood and embraced fully.

Background to this Report

The HSSFC report Opinion Poll on Academic Views of Electronic Publishing (Archer) -- a document that, with others in the report Scholarly Electronic Publishing in the Social Sciences and the Humanities in Canada, led to the commissioning of our study -- registers an irony, more direct and useful than that discussed above, in its notation of the seeming paradox that

many scholars have become active users of electronic resources in their scholarly research. However, they are still much less likely to try to publish their own scholarship electronically than they are to access materials electronically for scholarly purposes. (Archer 20)

Ultimately concluding that such reluctance can be directly attributed to issues surrounding credibility, Archer comments on the importance of professional peer review in this context, and offers a path towards a potential solution:

Clearly, faculty members in the Social Sciences and Humanities feel there is a difference in quality between these two types of publication [print and electronic]. . . . One obvious way of reducing this concern about the differences in quality is to adopt mechanisms of peer review similar to those used in quality scholarly publications. (8)

Here the study draws particular attention to the notion of perceived value and how it relates to academic publication. Central to the perception of value is the process of publication, imprint, and peer review. Well managed and understood in the print medium, the very nature of the electronic medium -- a new environment, in which clear and accepted standards are only now beginning to emerge -- has meant that these concerns have been treated with a skepticism appropriate to the academic arena.

Respondents to the survey -- academic teachers and authors, chiefly -- quite strongly rejected the notion that there is little difference between electronic and non-electronic publication (8). That said, they agreed that the adoption of appropriate peer review and archival mechanisms by those who publish academic material in electronic form would make electronic publication more acceptable (8-9). At heart is the issue of credibility, with some 85% of all respondents agreeing that "publishing in non-electronic outlets is more credible than publishing in electronic outlets" (9).

It is important to realize that the problem of establishing credibility is in one sense circular: because established scholars may be suspicious of the medium (even if they make use of resources disseminated via it), they are unwilling to publish in it; and because they do not themselves publish in it, all work published in that medium will continue to battle for credibility. Young scholars, in particular, are unlikely to take the risk of publishing in an electronic format. Archer comments on this problem:

A second concern is over the value that [is] ascribed to electronic scholarship. Most members of academic staff at Canadian universities go through a periodic (often annual) assessment of their academic performance, and for many, this assessment is tied to a process of awarding merit awards. A concern over the perceived value of an electronic publication will inevitably lead to decreased usage of this method of dissemination. (20)

A further concern signaled by this report is a perception that the electronic text is unstable and, consequently, unreliable. The report considers the reasons given by scholars for their reticence in publishing scholarship electronically:

One group of explanations can be referred to as concerns about preservation of scholarship. Respondents indicated that the uncertainty over the long-term availability of electronic resources dampens their enthusiasm towards its use. (20)

All those who publish are, at some level, concerned with archival issues, so it is hardly surprising to learn that Canadian academics are concerned with the long-term preservation and continued availability of their work. This concern is especially marked in relation to the electronic medium, which is both new and significantly different from print.

Thus, the survey conducted and reported on by Archer, as do others in the larger report (Electronic Publishing in the Social Sciences and the Humanities in Canada), paint a picture of the professional pragmatics that must be taken into account when considering scholarly publication in the electronic medium today.  These concerns see fuller elaboration in this document.

Project Description

In response to a call for targeted research (see Appendix E) a team assembled from among the faculty at Malaspina University-College, the University of Victoria, the University of New Brunswick, McMaster University and Université de Montréal had, in April of 2000, proposed and agreed (in May) to undertake an exploration of the urgent issues relating to the perception -- and ultimately the use -- of electronic publication within the Canadian academic community.  At the heart of the proposed study were issues surrounding the credibility of electronic publication that were identified by Archer and flagged by HSSFC call.  (Please see Appendix B for the research team's proposal.)

Drawn from among academics across the country in the Humanities and Social Sciences with expertise in a wide variety of pertinent areas, the research team offers not only considerable knowledge of and experience in Canadian scholarship, but also a significant amount of experience with electronic scholarly publication both in Canada and beyond. (Please see Appendix A for the composition of this team.)

The research team has provided a critical assessment of the North American and European literature surrounding the notion of credibility in electronic scholarly publication and has made recommendations that take into account both that literature and factors unique to our national context. Assessment and recommendations have been made in distinct, though interrelated, areas of Peer Review and Imprint, Copyright, and Archiving and Text Fluidity / Version Control.


 

Peer Review and Imprint

Peer review, identified by Archer's survey as the most important factor to assuage the reluctance of scholars to publish electronically, is a process that has evolved over many generations of scholars; it has become the cornerstone of academic publication and is something that is highly valued in all scholarly activities (among them, the pragmatics of academic review processes on which promotion and tenure are based).  It is important to recognize that peer review is necessarily, and appropriately, a conservative process, and that any new scholarly endeavor will take time to gain general acceptance.

The team -- with the leadership of Jean-Claude Guédon (U Montréal) in this area -- has located, selected, and provided a representative bibliographic overview of pertinent literature, literature that helps us identify the characteristics of peer review processes suitable for electronic publication, literature that helps us consider the implications of such processes both for reviewers and authors, and literature that recommends best practices that accord as much as possible with those already accepted by the academic community for non-electronic scholarly publication.

Like peer review, a publisher's imprimatur, or imprint, is seen to be a very important indicator of qualitative assurance in academic culture. As part of its study of peer review, the team has also treated the subject of imprimatur as related to that of peer review, and in a similar manner -- chiefly directing the reader to literature that proposes best practices aimed at both traditionally-print publishers operating the in the electronic medium and newer academic publishing groups that operate solely in the electronic medium.

Recommendations on the topic of peer review and imprint remain largely implicit, but a suggestive overview of current thought is given, as is an annotated bibliography and a summary of the data gathered on these topics via the questionnaire that was circulated among a representative sample of publishers, scholars, representatives of the Aid to Scholarly Publication Programme, reviewers for SSHRC and other bodies, university administrators and academic authors.

 

Copyright

Under the direction of Michael Best and Elizabeth Grove-White (U Victoria), the team has located, selected, and provided an overview of pertinent literature that highlights critical issues affecting the ownership of online resources, focusing on those resources that are text-based. The report also provides links to sites where the discussion of copyright is ongoing.

Work in this area, the report asserts, suggests that if copyright regulation is important for the traditional print publication mission of academic research institutions, it is doubly important in the context of new digital communication media, since international and national copyright principles and regulations were developed before networked digital media transformed the information and communication marketplace.

Broadly speaking, Canadian copyright legislation recognizes that copyright ownership rights belong to the creator of a work and those rights inhere in a work from the moment it is fixed in a tangible medium of expression.  But it is normal in the academic community for the author to cede copyright to the publisher; thus, it follows that most academic texts belong not to an individual but to a collective (a journal, a press).  

Academic authors are more concerned about the "currency" of the profession -- tenure, promotion, salary -- than about royalties or ownership of their work.  In those disciplines where the norm for scholarly interchange has been through monographs published by university presses, more often consulted in the library than bought individually by scholars or students, there  has been little challenge to the practice of ceding the ownership of scholarly work to the press. In the digital age, however, this practice is likely in due course to reduce substantially the potential readership of scholarly works, especially of journal articles.

At the same time as academic attitudes to copyright de-emphasize ownership issues that dominate the realms of the professional writer and press, they strongly underline the importance of wide promulgation of texts and transparent access for users.  Addressing copyright in the context of scholarly publication means addressing fundamental principles of academic culture, specifically the academic community's mission of advancing knowledge through creating, validating, and disseminating new knowledge, and through preserving the existing corpus of public knowledge.  Academic publication is central to these activities because it is the main process through which newly discovered knowledge is refined, certified, distributed and archived.  In addition to promulgating knowledge, publication provides the North American academy with the mechanisms for assessing the quality and quantity of contributions by individual faculty members that are used to determine faculty eligibility for tenure, promotion, grants and fellowships.

With the advent of the Internet and increasingly user-friendly technologies for online editing, publishing, and distributing available to faculty, an increasing number of disciplinary groups have turned to online journals and other online forums for disseminating their knowledge.  While the academic community continues to have reservations about this seemingly unruly new medium and the implications for the traditional academic activities for validating, authorizing and archiving new knowledge, the push for electronic forms of scholarly communication reflects a search for more timely, convenient, and economic means of announcing and certifying new research results.

But while the Internet is bringing steady and fundamental changes in the processes of scholarly communication and publication, these changes are occasioned less by the potential of the new media than by economic considerations occasioned by the changing world of academic publishing.  In response to this publication crisis, influential academic groups within Canada and beyond see digital publication as a strong alternative to the limitations of the current print environment and a non-commercial alternative to traditional scholarly communication. 

Because issues of copyright -- both ownership and fair use access -- have become so imbricated with perceptions of the economic potential in electronic publishing, questions about the distinctive operation, use, and unique copyright challenges posed by digital media have been poorly represented in these debates.  

Both the actual legislation concerning copyright, and the perception of intellectual ownership of data within the academic community are bound to evolve as the new medium both forces and encourages change.  It will be a major challenge in the next decade to find ways of making knowledge more freely available in electronic format, without reducing the legitimate needs of those who create content for a living.


 

Archiving and Text Fluidity / Version Control

Led by members of the University of New Brunswick Electronic Text Centre -- Alan Burk, James Kerr, and Andy Pope, a group that is currently involved in a larger study of journal metadata, citation linking and journal archiving -- the team has structured its review in this area to take into account a wide number of concerns associated with identifying and preserving academic work in the dynamic and evolving nature of the electronic medium.

While the internet is being populated with texts that are functionally indistinguishable from their print equivalents, it has also become a medium for new sorts of texts that do not fit well within the standard taxonomy of scholarly publication.  Academic web sites, and electronic scholarly texts, can incorporate traditional text, non-linear sequencing, various sorts of digital multimedia objects (image, audio, &c.), and a level of interactivity that goes far beyond the relation between reader and static print.  To read them may require specific hardware and software.  Given these new forms of publication, the archiving challenges are many. Also contributing significantly to the problem is how easy it is for individuals and groups to create or modify electronic publications and the variety of formats, some of which are proprietary, which are the carriers of the new electronic media.

Archiving

The report asserts that there are four processes that might preserve digital objects; they are as follows:

The literature describing these four processes or promoting one or another of them is complex and, necessarily, speculative.  The explicit speculation tends to focus on what is technically possible or administratively plausible.  The social and economic components of any digital preservation program are assumed more often than they are stated or examined.  No one knows what a national or international-level digital preservation program would cost or how its implementation would be structured.  However, institutions such as the National Library are trying to grapple with planning for a national level archiving and preservation program for the humanities and social sciences, looking at such issues as funding requirements, level of preservation, criteria for selecting objects to be preserved, and working over time with electronic objects where the original format and associated software have ceased to be generally supported. Their efforts are discussed in a forthcoming summary of an interview between the Electronic Text Centre and National Library staff, one of a series of interviews with key stakeholders.

Preservation of human artifacts has been largely a societal, rather than an individual or corporate, activity.  Funding for libraries, museums, and archives is implicitly contingent on their acceptance of various preservation responsibilities and the recognition by national and provincial level funding bodies of the need for digital preservation.  Digital preservation is different from traditional preservation in the sense that no agency is currently being funded to provide national level digital preservation.  The institutional will to create digital archives and the societal support to maintain them may develop, but if they do not, it really will not matter whether the technical problems raised by digital archives can be solved.

Adherence to standards is sometimes suggested as an additional method for digital preservation, but it is not a preservation process.   Standards are only as good as their level of social acceptance.  Even those that have wide acceptance may subsequently be rendered obsolete or irrelevant by changes in the external environment. 

No academic or research library aims to preserve the entire corpus of print publications.  Preservation of the scholarly record -- consisting of peer reviewed publications plus the indexing and abstracting tools to access this literature -- is generally considered essential to the role of academic and research libraries.  A pressing digital preservation problem is not in this mission but, rather, has to do with the incredible expansion over the last decade or so of the number and variety of digital objects that are not apart of the scholarly record, per se, but are cited within the scholarly record or might be the subject for future scholarly investigation.

Text Fluidity / Version Control 

The issue of text fluidity is a red herring as far as digital preservation is concerned.  A fluid is not fixed, firm or stable. A text is fixed, firm and stable, at least for the duration of any reading. The conjunction of the two concepts, "text fluidity," produces one of those jarring metaphors intended to enlighten but inclined to mislead and confuse; a more neutral and clarifying phrase is "version control."

Any modifications to a text, of any sort, produce distinct versions, all related to the original text; modifications to a text do not produce a text that is fluid.   Naming versions of a text and specifying the differences between the revision and the original text are the two elements of version control.  Were one to consider or preserve multiple versions of a given text, every version of that text could be uniquely named and its relation to the first version (and others) could be specified, if such relational specificity was warranted given the texts or objects involved.

Simply because electronic publications can be changed with ease does not mean that the canons of scholarly publishing could be or should be modified to accommodate large numbers of variants on a single text.  Scholarly publishing can be expected to change in response to the opportunities offered by electronic texts, but the change will be incremental.  

The most likely scenario for version control remains as it is now: acceptance of variant texts will depend on the type of text and the discipline. Refereed journal articles, scholarly monographs, “grey” literature, and secondary scholarly texts form a continuum with regard to the toleration of changes to the text, with refereed articles being the least tolerant.

Web sites can incorporate material that goes far beyond the relation between reader and static print. Academics who invest their time, skills and creativity in the development of such web sites need a reasonable expectation that their efforts will not be dismissed as just a new form of grey literature.  Implementing some simple conventions for version control, particularly in conjunction with greater use of metadata by web site creators, would facilitate the archiving process for innovative scholarly web sites and would constitute a starting point for gaining academic acceptance -- but attaining academic respectability will likely require much more than formal version control, including some mechanism for peer review and criteria that a site or collection of digital objects need to satisfy to be considered scholarly.


 

Methodology, Survey / Questionnaire, and Review

Methodology

As noted above, the team has based its observations and recommendations on an analytical review of the growing bodies of literature, national and international, that is related to the areas specifically identified by the HSSFC as being integral to the perceived credibility of electronic scholarly publication -- combined with useful materials beyond those related to our specific mandate, with attention paid to initiatives that have particular relevance to the Canadian academic community, and taking into data materials gathered by two surveys: one, discussed immediately below, which addressed issues of importance to all areas addressed by this report, and another, discussed in the above section on Archiving and Text Fluidity / Version Control, which addressed issues specifically related to that section of the report that necessitated specialized expertise.

Survey / Questionnaire

As per our proposal, members of the team conducted a survey with a representative number of publishers, scholars, representatives of the Aid to Scholarly Publication Programme, reviewers, university administrators, and academic authors -- via a questionnaire; this survey has illuminated the international literature review, and has helped us to situate the results of that literature review specifically within the Canadian context that this report seeks to serve.

The questionnaire was designed with some consultation with people with experience in quantitative data gathering and analysis; its contents also reflect input into the full range of concerns addressed by this report.  Specifically reporting on this survey are Geoffrey Rockwell (McMaster U) and, via data analysis, Lynne Siemens (Malaspina U-C); the questionnaire itself was designed jointly by the research team, first collated and administered under the leadership of Joanne Buckley (McMaster U), then developed further and its delivery overseen by Ray Siemens (Malaspina U-C), Michael Best (U Victoria), and a team based at both the University of Victoria and Malaspina University-College.

For the questionnaire itself, in both official languages, please see Appendix C1 and Appendix C2.  For the report on the questionnaire, and summary of results, please see Section 5 of this report; and for a full analysis of the data collected by the questionnaire, please refer to Appendix D.

Review

As well, the research team carried out a review of the report and its findings before its final submission the HSSFC.  Composed of the following people -- 

-- this group has kindly volunteered their time for such a review.  We also wish to thank David McCallum (Canadian Association of Research Libraries), who was able to offer his thoughts and expertise.  Every attempt has been made to incorporate, accommodate, and note the comments, suggestions, and criticisms of this group in our report.


Acknowledgements

A project such as this one necessarily involves the assistance and support of a group much larger than that made up by the research team alone.

A large community of people in the academic and academic publications' community laid a firm foundation for -- and answered specific questions from us related to -- our study; the firmest of these foundations was laid by the working groups responsible for the HSSFC report Electronic Publishing in the Social Sciences and the Humanities in Canada.

In addition, an HSSFC committee took the time to make comments on the research focus proposed by our team, comments that ensured that our work would meet best the needs of our community.  Louise Robert, Simon Lapointe, and others at HSSFC offered valuable assistance, as did Yves Mougeot, Katharine Benzekri, and others at SSHRC, as did those who acted as reviewers for our report (noted above), and as did (anonymously) those many academics, administrators, and publishers who took time away from other projects to respond to our questionnaire.

Beyond this already large group, for general assistance and specific work with the section on Peer Review and Imprint, we would like to thank Laura Johnston (Malaspina U-C); for their assistance in the preparation and delivery of the questionnaire, we would like to acknowledge the early leadership of Joanne Buckley (McMaster U), and the assistance of William Winder and Ruth Hanson (U British Columbia), Tana Morgan and Elise Mitchell (U Victoria), Jean Blackburn (Malaspina U-C) and others at Malaspina U-C's Educational Technology Centre.

The research team would also like to thank their home institutions for their generosity and assistance offered while work on this project was being carried out -- and, perhaps most importantly, the HSSFC, for giving us the opportunity to make a positive contribution in an area very important to all our related disciplines.


Works Cited


12/05/2000