Comparing information literacy of student beginners among different branches of study

In our paper we present a study in which we investigated the level of information literacy of first-year students across different studies offered by faculties at the University of Graz. Data were collected by means of a multiple-choice questionnaire. In total we analyzed 232 questionnaires completed by first-year students from six studies (faculties). The results show that the overall level of information literacy is mediocre. However, the difference in the level of information literacy across the studies is not as big as originally expected. The type of high school completed has a strong effect at least in the beginning of the studies. Another interesting result shows that the students rated their information literacy skills much higher than the test indicated. It can be concluded that multiple-choice questionnaires are an efficient and objective test instrument to inform the students that they are much less information literate than they believe themselves to be.


Introduction
In recent years information literacy has become a key competence.Skills, such as searching, assessing and using information are not only required of university students but, in principle, of each participant of the knowledge society (European Parliament 2006, UNESCO andIFLA 2012).To describe this process of dealing with information several models have been developed in the past two decades (e.g.Eisenberg a Berkowitz 1990, Kuhlthau 1993, SCONUL 2011).To support the teaching of information literacy as well as the assessment of information literacy skills additional standards have been developed.Probably the best known are the "Information Literacy Competency Standards for Higher Education" (ALA 2000).In response to the dynamic information ecosystem and the rapidly changing higher education environment, the board of the Association of College & Research Libraries (ACRL) adopted the Framework for Information Literacy for Higher Education in 2016.The term "framework" was chosen intentionally because "… it is based on a cluster of interconnected core concepts, with flexible options for implementation …" (ACRL 2016, 2).A growing awareness of information literacy does not automatically mean that these skills can be taken for granted.Though nowadays everyone knows how to get quick search results using search engines such as Google, most of the users have no idea that there exist alternative information sources of much higher quality.Most Google users are not familiar even with many of the advanced search features of Google.Accordingly, search engines may even affect information literacy since their users believe that they are more information literate than they actually are (Virkus 2003, Selwyn 2009, Ferrari et al. 2012).

Research questions and methodology
Multiple-choice tests are one instrument that can be used to determine the level of students' information literacy.First studies with this kind of information literacy tests have already been conducted.One such study, involving thirdsemester students enrolled in the bachelor degree programme in Business Administration, revealed that the level of information literacy was low to average at the beginning of the course.Furthermore, the acceptance of the test instrument was assessed to be good.18 out of 27 students found it "reasonable", while three found it "very reasonable" (Beutelspacher, Henkel and Schlögl 2015).In this follow-up study we want to extend our previous analysis to different branches of study.Hence, the main goal is to identify the level of information literacy among first-year students from different studies.
On this basis the following four research questions are addressed: • RQ 1: What is the level of information literacy of first-year students?
• RQ 2: Is the level of information literacy comparable across different branches of study?
• RQ 3: Does the level of information literacy depend on the type of high school completed?
• RQ 4: Do students evaluate their information literacy skills higher than the information literacy test indicates?Data were collected by means of a multiple-choice questionnaire partly based on existing test instruments (Beutelspacher 2014, Mittelmeyer and Quirion 2003, Gerharter 2015).However, many questions have been changed, several have been omitted and a few new ones included.Finally, the questionnaire consisted of three main parts: The questions in the first part ask for personal data, such as gender, age, type of high school completed, and name of studies.Furthermore, students could choose among several pre-defined information resources they use and indicate how often they use them in their studies.In part two, the students had to evaluate their level of information literacy.For this purpose, they had to give a self-assessment (1=very good, … 5=very bad) on five different aspects of information literacy.The main part of the questionnaire included 27 single and multiple-choice questions which aimed at identifying the information literacy proficiency of the students.While the Information Literacy Framework "…seeks to address the great potential for information literacy as a deeper, more integrated learning agenda …" (ACRL 2016, 12), our study is more basic aiming at checking fundamental information literacy elements of student beginners.As a consequence, we based the design of the proficiency test on parts of the following four Information Literacy Competency Standards for Higher Education (ALA 2000): • ability to determine the extent of information needed (standard 1) • ability to access the needed information (standard 2) • ability to evaluate information and its sources (standard 3) • ability to understand the economic, legal, and social issues surrounding the use of information (standard 5).
Since these information literacy standards are well operationalized, they are also very supportive when designing a proficiency test.
In order to have a high response rate, we decided not to perform an online survey.Instead, we contacted the course instructors of six first semester courses and asked them if they would allow data collection either at the beginning or at the end of one of their classes. 1Each of the six courses -"Analysis 1", "Introduction to Linguistics", "Mathematics for Business Administration", "Selected Topics in Private Law, Public Law, Criminal Law", "The History and the Disciplines of Theology", and "Theory of Education I (Basics and Problems of Educational Theories)" -is offered by a different faculty of the University of Graz ("Natural Sciences", "Arts and Humanities", "Business, Economics and Social Sciences", "Law", "Catholic Theology", and "Environmental, Regional and Educational Sciences").Data collection took place in December 2015.In one course the questionnaires were distributed in January 2016.Usually it took the students 20 minutes to complete 1 It is highly impressive that all contacted colleagues gave their approval to conduct the test in their course.Many thanks for the support.
the test 2 .We included one bachelor programme from each faculty (Mathematics from the Faculty of Natural Sciences; German Language and Literature Studies from the Faculty of Arts and Humanities; Business Administration from the Faculty of Business, Economics and Social Sciences; Law; and Catholic Theology from the correspondent faculties; and Educational Sciences from the Faculty of Environmental, Regional and Educational Sciences) in our analysis, resulting in 232 analyzed questionnaires.
Based on the given (single or multiple-choice) answers, an information literacy score was calculated.The highest possible score was 31.5 points.On the basis of this score, marks ranging from 1 (best mark) to 5 (worst mark) were assigned (for details see Maurer 2016).

Results
Table 1 shows the distribution of the 232 students across the faculties.Most of them come from the Faculty of Arts and Humanities (52 students) and from the Faculty of Environmental, Regional and Educational Sciences (51 students).The small number of participating students from the Faculty of Catholic Theology (14 students) can be explained by the fact that this is by far the smallest faculty at the University of Graz.
The gender distribution varies across faculties and studies.Female students constitute the majority at the Faculty of Arts and Humanities (77%) as well as in Educational Sciences offered by the Faculty of Education, Regional and Environmental Sciences (82%).In the other studies the gender bias in the sample was not so pronounced.In total 63% of the participating students were female and 37% were male.

Level of information literacy of first-year students
As can be seen in Figure 1, the test results for the 232 students were not as bad as originally expected.The assessments ranging from 1 to 5 are nearly normally distributed.While only seven students (3%) got a top assessment, 24 students (10%) failed (were graded with 5).The numbers of students who have passed the test with grades 2 or 4 were almost equal: 51 and 52 students respectively (22% each).Most students (98 or 42%) completed the test with an average assessment (mark 3).

Comparing the level of information literacy across different studies
When comparing information literacy across the six branches of studies (see Tables 3a and 3b), it must be considered that the study programmes (faculties) vary greatly in the number of enrolled students.Since only 14 questionnaires could be collected from first semester students of "Catholic Theology", their test results may not be as representative as those obtained for the other studies.
The differences across the six faculties are not as large as originally expected.Catholic Theology students proved to have a slightly higher level of information literacy than the students of German Language and Literature Studies from the Faculty of Arts and Humanities and Mathematics students from the Faculty of Natural Sciences.Slightly lower scores were obtained by the Law students and the Educational Science students from the Faculty of Environmental, Regional and Educational Sciences.Business Administration students from the Faculty of Business, Economics and Social Sciences were found to have the lowest level of information literacy.Nearly half of them earned the two lowest grades (4 or 5).

Comparing the level of information literacy across different types of high schools
At the beginning of a university study the level of information literacy might be influenced by the type of high school completed.As can be clearly seen in Table 4, students who have completed a general-education high school were found to be more information literate than students from a vocational high school.While one third of the students who completed a general-education high school were given the two highest marks (1 or 2), only 15 percent of the students with a school leaving examination from a vocational high school were rated as Good (2).Two students who had passed a vocational matriculation examination failed the information literacy test, one was given the grade 3.One student who has completed a University Admission Examination was given the grade 4. Furthermore, the fact that most Business Administration students (54%) come from a vocational high school could explain the lower level of information literacy skills acquired in this type of high school.

Comparing information literacy self-evaluation with the actual test results
As already mentioned, the students had to assess their level of information literacy with regard to the following five aspects: • ability to identify their information need • overview of relevant information resources • information searching on the Internet • evaluation of information and its resources • knowledge of economic, legal, and social issues surrounding the use of information.
As evidenced in Table 5, there is a significant difference between selfperceived information literacy skills and the actual level of information literacy according to the test results.This is in particular true for information search skills where nearly half of the students (48%) thought that they were very good (mark: 1).A marked divergence can also be observed with regard to the ability to identify one's information needs where 81% of the students assessed their skills as "very good" (1) or "good" (2), and the ability to evaluate the quality of information and information resources which more than two thirds assessed as "good".Only in the case of the knowledge of economic, legal, and social issues, the self-evaluation was closer to the actual information literacy test results, according to which most students (42%) were graded satisfactory (3) but only three percent of them were marked "very good" (1).

Other results
In part 1 of the questionnaire the students were asked to indicate their Internet search preferences and information resources they use as well as how often they use them in their studies.It is not surprising that web search engines are used several times a day by the great majority of the students (77%).Slightly more than one tenth of the students use web search engines "only" several times a week.
Web search engines and electronic media are also frequently used for study purposes by the majority of the students.As can be seen in Figure 2, Google and Wikipedia are by far most frequently used: always -71% (Google) / 30% (Wikipedia), often -26% (Google) / 56% (Wikipedia).Other web search engines are seldom (36%) or never used (44%), which shows the quasi-monopoly of Google in the European search engine market.Other wikis as well as blogs and web forums are often or always used by half of the students.This contrasts strongly with the use of "classical" information resources -books (from the library) and print journals -which are frequently (often and always) used by less than one fifth of the students.Finally, e-journals are used by a great proportion of students often (32%) or always (5%).Library catalogues and factual and literature databases are unknown to 7% and 13% respectively of the students.What causes even greater concern is that they are never used by 41% and 51% of the students.Only 13% of the first-year students use library catalogues often and only 2 % always.These proportions are even smaller with regard to the use of databases (10% -often, 0% -always).

Discussion and conclusions
Our study shows that the level of information literacy of first-year students at the University of Graz is mediocre.One tenth of the students even failed the test, i.e. they were not able to correctly answer at least half of all single and multiple-choice questions.The difference in information literacy across the five faculties was not as big as originally expected.The level of information literacy was highest for students of Catholic Theology and German Language and Literature.One explanation could be that these studies are much more literature/book-related.Accordingly, first-year students brought a higher level of information literacy to their studies.Business Administration students were found to have the lowest level of information literacy since half of them received the lowest marks (4 and 5).The high number of first-year students is the result of Business Administration being one of the most popular studies at the University of Graz.It involves massive courses in the first semester that do not require individual scientific research.Another explanation could be that most Business Administration students have completed a vocational high school before their university study.The test has also revealed that the level of information literacy is clearly higher for students who have completed a general-education high school.
Another interesting result of our study is that students' self-assessed level of information literacy is much higher than it actually is.This is particularly true for information search skills which were assessed as very good (1) by nearly half of the students and good (2) by 40% of the students.This differs considerably from their actual level of information literacy.For instance, 41% of the students indicated that they never used library catalogues, half of them did not use databases, 13% were not even aware of them.These results clearly demonstrate the importance of extending students' information literacy knowledge and skills at the beginning of their university studies.
We are aware that we cannot evaluate high-order information literacy skills with our test instrument (Scharf et al. 2007).However, in order to get an idea of (non)existing information literacy skills, it might be sufficient to assess the gap in information literacy of first-year students and to show them that they are much less information literate than they believe themselves to be.Multiplechoice questionnaires are suitable for this purpose.They are an efficient and convenient knowledge assessment tool which yields highly comparable results (Beutelspacher, Henkel and Schlögl 2015).

Figure 2 .
Figure 2. Frequency of use of various information resources

Table 1 .
Number of participating students by faculty and gender (n=232) Austria has a diversified high school system.Essentially, high schools can be divided into those offering general education and those providing vocational education.Both are completed by taking a school leaving examination which 2The test instrument can be downloaded from INWI (2016).Alica Kolarić,

Ivanka Stričević, Information seeking behavior for decision making in everyday life: a pilot study on adolescents,
Libellarium, IX, 2 (2016): 275 -308enables the pupils to enrol in a full-time university study.Alternative ways include the successful passing of a university admission examination and a vocational matriculation examination.As can be seen in Table2, most students have completed a general-education high school (61%), while 37% have finished a vocational high school.

Table 2 .
Number of participating students by type of university admission qualification (n=232)

Table 3a .
Level of information literacy by branch of study (faculty) -part 1

Table 3b
. Level of information literacy by branch of study (faculty) -part 2