CAMEO | Ask A Librarian |  Site Search | Home

SAILS Description

SAILS is a knowledge test for undergraduates with multiple-choice questions targeting a variety of information literacy skills. The test items are based on the ACRL Information Literacy Competency Standards for Higher Education. It was developed at Kent State University. More than 80 institutions and 42,000 students have participated in SAILS as of the fall 2006 administration.

SAILS at Carnegie Mellon

Carnegie Mellon University administered SAILS as a diagnostic tool. If participation is high enough to be significant, the data gathered by the test will be used as a baseline measurement of undergraduate students' information literacy skills. This data may help direct future information literacy programming at the university. This trial administration was also used as an assessment of the test itself. If the SAILS test is deemed a good fit for assessing Carnegie Mellon students' information literacy skills it may set a precedent for future SAILS administrations at Carnegie Mellon.

The test was administered on the web at http://www.projectsails.org, and in paper format by request. The test administration ran from late September 2006 until December 2006. The Carnegie Mellon administration of the SAILS test is a unique one in that it was administered in an unmonitored, completely online way. While there are paper tests, students were encouraged to take the test online at their leisure. This method of administration was chosen because of the large number of students to be tested. 5,620 emails with unique student identifiers were sent out in late September to every undergraduate student. Email reminders were sent throughout the semester.

Results

Three hundred and sixty two Carnegie Mellon undergraduates completed the SAILS test. Demographics information indicating class year, school/college, major, and native language was collected. The demographics roughly align to Carnegie Mellon's undergraduate body; however, freshmen and MCS students were slightly over represented while CFA and Tepper students were slightly underrepresented. Twenty two percent of Carnegie Mellon's SAILS participants self identified as non-native English speakers.

The SAILS test is scored using an item response theory called the Rasch Model. In a nutshell, this model ranks each test item based on past student performance and then places those scores on a scale from 0 to 1000 with 500 being the mean score. Each 100 points indicates a standard deviation. A comparison of Carnegie Mellon's performance against all other Doctoral/Research Institutions within each SAILS skill set looks like this:

It should be noted that standard errors for Carnegie Mellon's scores are large because of the relatively small sample size of Carnegie Mellon's administration compared to the 42,000 nationwide SAILS participants. Standard errors for the Carnegie Mellon administration ranged from ±10 to ±20 for all skill sets.

What does this mean?

Overall, Carnegie Mellon students performed very well on the SAILS test. For example, Carnegie Mellon students performed better than the Doctoral/Research institutions benchmark on Developing a Research Strategy, Using Finding Tool Features and most other skill sets. Selecting Finding tools was lower on the performance list so the University Libraries may focus instruction efforts on this skill set.

One must keep in mind some caveats when interpreting these results. First, participants self-selected to take this test, meaning these students may have had a preexisting aptitude for some or all of the skills assessed by the SAILS test. On the other hand, according to Project Sails, students tend to do less well when they take the test in an unmonitored setting, as was the case at Carnegie Mellon. With these factors in mind, the University Libraries plans to incorporate the general findings from SAILS into our continually developing information literacy instruction activities.

These results will be reported to the University Education Council this spring, 2007.

Contact Information:
Jean Alexander - Head of Hunt Reference
jeana@andrew.cmu.edu, 412.268.6809

Dan Hood - Information Literacy Fellow
dhood@andrew.cmu.edu, 412.268.2536