Climate Survey

Rankin Climate, LLC, which is serving as the outside consultant for the Antioch University climate survey, defines university climate as “the current attitudes and behaviors of faculty, staff, administrators, and students, as well as institutional policies and procedures, which influence the level of respect for individual needs, abilities and potential.” The climate is often shaped through personal experiences, perceptions, and institutional efforts. The Climate Survey is a tool to assess the climate at Antioch. Findings from the survey will enable University leaders to take actions that improve the experiences of community members and to chart a path to achieving the University mission.

Rankin Climate’s research maintains that positive personal experiences with university climate and positive perceptions of climate generally equate to successful outcomes. Example successful outcomes include positive educational experiences and healthy identity development for students, productivity and sense of value for faculty and staff, and overall well-being for all.

The idea to conduct a climate survey originated from interested students, faculty, and staff who believed data from such a survey might be useful in planning for the future and improving the climate at Antioch University. It is an effort to generate data-driven insights that the University can use to become more equitable and effective. Examples of successful outcomes include positive educational experiences and healthy identity development for students, productivity and a sense of value for faculty and staff, and overall well-being for all.

The Climate Survey Working Group (CSWG) is charged with conducting Antioch University’s climate survey. After a review of potential vendors, the committee selected Rankin Climate to conduct the survey. Rankin Climate reports directly to the committee. Although the CSWG will regularly update Antioch University about its progress, the committee—in consultation with Rankin Climate—is solely responsible for the development, implementation, and interpretation of the survey and its results. Rankin Climate’s Dr. Stefani Bjorklund, Dr. Stephanie Danette Preston, and Dr. Gabriel Reif will be the associates working directly with us on this project. Rankin Climate has conducted multi-location institutional climate studies at more than 250 institutions across the country. Rankin Climate developed and uses the Transformational Tapestry model as a research design for university climate studies. The model is a “comprehensive, five-phase strategic model of assessment, planning, and intervention. The model is designed to assist university communities in conducting inclusive assessments of their institutional climate to better understand the challenges facing their respective communities” (Rankin & Reason, 2008).

In reviewing efforts by other universities to conduct comprehensive climate studies, several best practices were identified. One was the need for external expertise in survey administration. The administration of a survey relating to a very sensitive subject like climate is likely to yield higher response rates and provide more credible findings if led by an independent, outside agency. Members of a college community may feel particularly inhibited to respond honestly to a survey administered by their own institution for fear of retaliation.

Submitting the survey to the university’s Institutional Review Board (IRB) is an important step of the climate study project. There are several benefits to going through this process, even if the survey is considered to be “exempt” or “not human subjects research.” Firstly, it provides some assurance to potential participants that their responses to the survey will remain confidential. People who have participated in social science research may recognize and expect that the process has been reviewed by IRB. They may feel more comfortable sharing their confidential information and personal stories. Secondly, going through an IRB review provides some protections to the Principal Investigators (PIs) who will be entrusted with analyzing and managing the data, including both R&A and the institutional PIs. The expectation is that the data will not be shared with others, as designated in the data security plan. Lastly, an IRB review provides some cache to the study in the eyes of those who conduct research professionally, such as faculty members and scientists. These individuals understand what an IRB review requires and the protections for the data to which research professionals commit.

The Antioch University primary investigator for the IRB process is Dr. Teresa Kaldor, University Director of Institutional Effectiveness. An IRB application will be submitted for the project. Once the project is approved, the survey will be administered. An IRB application will be submitted for the project. Once the project is approved, the survey will be administered.

Although the committee believes the survey process itself is informative, we have sought and received commitment from the senior leaders that data will be used to plan for an improved climate at Antioch University.

The target participation in the survey is all students, faculty, and staff at Antioch University. Every response matters and is valuable in providing the most beneficial feedback and results.

Confidentiality is vital to the success of university climate research; particularly as sensitive and personal topics are discussed. While the survey cannot guarantee complete confidentiality because of the nature of multiple demographic questions, the consultant will take multiple precautionary measures to enhance individual confidentiality and the de-identification of data. No data already protected through regulation or policy (e.g., Social Security number, campus identification number, medical information) is obtained through the survey. In the event of any publication or presentation resulting from the assessment, no personally identifiable information will be shared.

Confidentiality in participating will be maintained to the highest degree permitted by the technology used (e.g., IP addresses will be stripped when the survey is submitted). To avoid interception of data, the survey is run on a firewalled web server with forced 256-bit SSL security. In addition, the consultant and college will not report any group data for groups of fewer than five individuals, because those “small cell sizes” may be small enough to compromise confidentiality. Instead, the consultant and the college will combine the groups or take other measures to eliminate any potential for demographic information to be identifiable. Additionally, any comments submitted in response to the survey will be separated at the time of submission to the consultant so they are not attributed to any individual demographic characteristics. Identifiable information submitted in qualitative comments will be redacted and the college will only receive these redacted comments.

Participation in the survey is completely voluntary, and participants do not have to answer any question and can skip any other questions they consider to be uncomfortable. Paper and pencil surveys are also available and will be sent directly to the consultant.

Information in the introductory section of the survey will describe the manner in which confidentiality will be guaranteed, and additional communication to participants will provide expanded information on the nature of confidentiality, possible threats to confidentiality and procedures developed to ensure de-identification of data.

The consultant will provide a final report that will include: an executive summary; a report narrative of the findings based on cross tabulations selected by the consultant; frequencies, percentages, means, and standard deviations of quantitative data; and content analysis of the textual data. The reports provide high-level summaries of the findings and will identify themes found in the data. Generalizations for populations are limited to those groups or subgroups with response rates of at least 30%. The committee will review draft reports and provide feedback to the consultant prior to public release.

Rankin Climate uses a research data security description and protocol, which includes specific information on data encryption, the handling of personally identifiable information, physical security, and a protocol for handling unlikely breaches of data security. The data from online participants will be submitted to a secure server hosted by the consultant. The survey is run on a firewalled web server with forced 256-bit SSL security and is stored on a SQL database that can only be accessed locally. The server itself may only be accessed using encrypted SSH connections originating from the local network. Rankin & Associates Rankin & Associates will have access to the raw data. All Rankin & Associates analysts have CITI (Human Subjects) training and approval and have worked on similar projects for other institutions. The web server runs with the SE-Linux security extensions (that were developed by the NSA). The server is also in RAID to highly reduce the chance of any data loss due to hardware failure. The server performs a nightly security audit from data acquired via the system logs and notifies the administrators. The number of system administrators will be limited and each will have had required background checks.

The consultant has conducted more than 250 institutional surveys and maintains an aggregate merged database. The data from the Antioch University project will be merged with all other existing climate data stored indefinitely on the consultant’s secure server. No institutional identifiers are included in the full merged data set held by the consultant. The raw unit-level data with institutional identifiers is kept on the server for six months and then destroyed. The paper and pencil surveys are returned to the consultant directly and kept in a locked file drawer in a locked office. The consultant destroys the paper and pencil responses after they are merged with the online data. The consultant will notify the committee chairs of any breach or suspected breach of data security of the consultant’s server.

The consultant will provide the principal investigator with a data file at the completion of the project.

The survey will be administered to all students at Antioch University. Climate exists in micro-climates, so creating opportunities to maximize participation is important as well as maximizing opportunities to reach minority populations. Along these lines, the consultant has recommended not using random sampling as we may “miss” particular populations where numbers are very small (e.g., Native American students). Since one goal of the project is inclusiveness and allowing invisible “voices” to be heard, this sampling technique is not used. In addition, randomized stratified sampling is not used because we do not have population data on most identities. For example, Antioch University collects population data on gender and race/ethnicity, but not on disability status or sexual orientation. A sample approach could miss many groups.

This initiative will include five phases, including survey development (Spring 2023), survey administration (October 2023), data analysis (November 2023 – January 2024), and report development and presentation (February 2023-April 2024). The final phase involves developing of strategic initiatives/actions and an accountability roadmap based on the findings (late Spring/Summer 2024).