Resources for Educational Research - Teaching Academy | College of Medicine | University of Vermont

Resources for Medical Education Research and Scholarship

GETTING STARTED: WHAT IS MEDICAL EDUCATION SCHOLARSHIP AND IS MY WORK SCHOLARLY?
Resource	Description	Link
Medical education scholarship: An introductory guide (AMEE Guide No. 89)	An overview of medical education scholarship for early career scholars, including the development of scholarly questions and a scholarly plan, guidelines for assessing the impact of scholarship, and dissemination strategies. This guide also includes career advice based on the authors' experiences.	Link to resource
The research compass: An introduction to research in medical education (AMEE Guide No. 56)	An introduction to medical education research intended for those who are new to the field. It is structured around the process of transforming ideas and problems into researchable questions, choosing a research approach that is appropriate to the purpose of the study, and considering the individual researcher's preferences and the contextual possibilities and constraints.	Link to resource
Educational Scholarship Guide	Describes the definition, peer review, publication, and recognition of educational scholarship in medical and dental education. Drawing on the educational scholarship literature, the documents illustrate how published educational works are comparable to other forms of scholarship that are commonly used for promotion and tenure purposes.	Link to resource

CONCEPTUAL FRAMEWORKS IN MEDICAL EDUCATION SCHOLARSHIP (SEE ALSO: THEORIES OF LEARNING)
Conceptual frameworks to illuminate and magnify	This paper outlines three examples of how conceptual frameworks can be used to provide perspective on research projects in medical education, and provides key points about the use of conceptual frameworks in scholarship.	Link to resource
Building bridges between theory and practice in medical education using a design-based research approach (AMEE Guide No. 60)	This paper discusses design-based research (DBR) and how DBR can be used to reveal findings that are of broader relevance than the local situation and how to insure that DBR contributes toward theory testing and refinement. Examples are provided to illustrate how DBR studies actually work in practice.	Link to resource

DEFINING SCHOLARSHIP IN MEDICAL EDUCATION
Boyer's Expanded Definitions of Scholarship, The Standards for Assessing Scholarship, and the Elusiveness of the Scholarship of Teaching	From Boyer, scholarship has four separate yet overlapping meanings: scholarship related to discovery, integration, application, and teaching. Six criteria for judging the quality of the scholarship of teaching are presented.	Link to resource
Evaluating educators using a novel toolbox: applying rigorous criteria flexibly across institutions	This paper describes a resource based on explicit evidence-based criteria to evaluate faculty in each of the five domains: teaching, learner assessment, curriculum development, advising and mentoring, and educational leadership/administration.	Link to resource
Using the AAMC Toolbox for Evaluating Educators: You be the Judge!	These resources are designed for use by promotions/tenure committees to apply the indicators in the AAMC Toolbox for Evaluating Educators to assess the performance of faculty members whose career focus is in education. Faculty and their mentors can also use them as a means for guidance and reflection about educational scholarship.	Link to resource

DESIGNING A RESEARCH STUDY
Experimental study design and grant writing in eight steps and 28 questions	This paper describes an eight-step, 28-question, iterative approach is proposed to help with the careful planning of experiments in order to maximize the researchers' chances of acceptance when submitting the study for funding and its results for publication.	Link to resource
Classifying basic research designs	This paper describes a three dimensional typology useful for describing the basic design qualities of any study. This typology may be helpful for teaching basic research design concepts and for research design decisions in planning a study.	Link to resource
Avoiding Five Common Pitfalls of Experimental Research in Medical Education	The authors present a worksheet outlining common research design problems along with suggestions about how to avoid them.	Link to resource
A Guide to Research Paradigms Relevant to Medical Education	In order to design or interpret qualitative and quantitative research, one should have some understanding of the assumptions that underpin them. The authors provide an overview of the concepts underlying four philosophical paradigms in medical education research and their inter-relationships.	Link to resource

EVALUATING EDUCATIONAL PROGRAMS
Teaching Academy Survey Question Development	This resource developed by the Teaching Academy discusses how to craft the ideal survey questions.	Link to resource
Common Evaluation Designs in Medical Education I	This resource describes three common program evaluation designs, as well as their benefits, limitations and examples.	Link to resource
A Snapshot of Three Common Program Evaluation Approaches for Medical Education	Program evaluation is described from three different approaches: "decision-oriented," "outcomes-oriented," and "expert-oriented." These perspectives are compared across five dimensions.	Link to resource
Evaluating educational programs: A planning tool	This is an introductory level web-based tutorial that guides users through the six steps to developing an evaluation plan for their program or curriculum. Examples from published programs are provided for each of the steps.	Link to resource
Evaluating educational programmes (AMEE Guide No. 29)	This guide provides a framework for planning an evaluation. The role of the evaluator; the ethics of evaluation; choosing the questions to be asked; evaluation design, including the dimensions of evaluation and the range of evaluation approaches available to guide evaluators; interpreting and disseminating the findings; and influencing decision making are covered.	Link to resource
Measuring research: A guide to research evaluation frameworks and tools	This extensive report describes the key considerations in developing an evaluation plan, outlining important trade-offs and contextual factors. A detailed overview of six research evaluation frameworks is provided.	Link to resource

HUMAN SUBJECTS AND IRB APPLICATIONS
Michigan State University Human Research Protection Program	The Michigan State University (MSU) Human Research Protection Program (HRPP) includes the Institutional Review Board (IRB) and the HRPP Compliance offices. The primary mission of MSU HRPP is to protect individuals who are the subjects of research. MSU is committed to follow the ethical standards described in the Belmont Report, and all applicable federal, state and local regulations and university policies and procedures.	Link to resource
Education Research and Human Subject Protection: Crossing the IRB Quagmire	The author provides an overview of current regulations that impact educational research and guidance for when IRB review is required.	Link to resource
Twelve tips for ethical approval for research in health professions education	The authors provide advice about the ethics application process for those conducting research in health professions education. The tips provided will help faculty identify, and devise plans to address, some ethical issues that are common in health professions education research.	Link to resource
When Do Medical Students Become Human Subjects of Research? The Case of Program Evaluation	This paper discusses the confusion around when student data from program evaluation or learner assessments subsequently become the basis for faculty scholarship.	Link to resource
Medical students as human subjects in educational research	This study explores the perceived level of risk when medical students serve as human subjects, comparing the responses to the same study protocol of Institutional Review Boards (IRBs) at seven institutions.	Link to resource

JOURNALS THAT PUBLISH MEDICAL EDUCATION RESEARCH AND SCHOLARSHIP
Annotated bibliography of journals for medical education scholarship	This bibliography provides information about over 30 journals that publish scholarship in medical education as well as other publication resources.	Link to resource

LOGIC MODELS
Using an outcomes-logic-model approach to evaluate a faculty development program for medical educators	This study describes the use of an outcomes-logic-model approach to examine the impact of participating in a nontraditional professional development program.	Link to resource
Implementing the logic model for measuring the value of faculty affairs activities	This article describes use of the program logic model to measure the contribution of faculty affairs and development offices to the recruitment, retention, and development of a medical school's teaching faculty, an outcome central to the mission of the medical school.	Link to resource
Logic Model Development Guide	This guide, developed by the W.K. Kellogg Foundation provides a practical orientation to the underlying principles of "logic modeling" as a tool to enhance their program planning, implementation, and dissemination activities.	Link to resource
Logic Models	This interactive website provides multiple resources to assist in the creation of logic models to guide program development, implementation and evaluation.	Link to resource
Program evaluation of a model to integrate internationally educated health professionals into clinical practice	This paper describes a program developed using a logic model and evaluated using program evaluation methodology.	Link to resource

PLANNING YOUR MEDICAL EDUCATION SCHOLARSHIP
How to succeed as a medical education scholar: Identifying your individual strategy and creating a roadmap for scholarship	This planning tool and worksheet provides a structure for working through the steps needed to successfully conduct and disseminate educational scholarship.	Link to resource

POSTER PRESENTATIONS
Creating effective poster presentations (AMEE Guide no. 40)	This guide focuses on posters as a means of visual communication of your scholarship and to engage an audience with a poster. This guide is supplemented with annotated examples.	Link to resource
Creating Posters in PowerPoint: A Video Tutorial	This 10-part video tutorial will show you how to make a poster using PowerPoint. Posters are used to show highlights of a project so that people who read it can quickly and easily understand it. Successful posters stand out, clearly communicate their content, are visually interesting and it must look like it can be understood in a few minutes.	Link to resource

QUALITATIVE RESEARCH
Standards for reporting qualitative research: A synthesis of recommendations	The study describes a review of the literature to formulate and define standards for reporting qualitative research recognizing the need for flexibility to accommodate various paradigms, approaches, and methods.	Link to resource
An introduction to reading and appraising qualitative research	This article introduces several major qualitative research approaches an how they differ in terms of: in the tools that qualitative researchers use, in the methodologies they use to analyze those tools, in the theories that inform those methodologies, and in their beliefs about knowledge itself. In addition, definitions of common terms are provided.	Link to resource
Grounded theory in medical education research	The paper discusses the use of the grounded theory method in medical education research and how to apply such a method to research questions. The important features of a grounded theory as well as its implications for medical education research are explored.	Link to resource
Grounded theory in medical education research (AMEE Guide No. 70)	The guide provides a historical perspective on grounded theory, and the principles underlying the approach and procedures for doing a grounded theory study. Key critiques of grounded theory, as well as pitfalls and controversies in grounded theory research are examined.	Link to resource
Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups	The criteria included in this 32-item checklist can help researchers to report important aspects of the research team, study methods, context of the study, findings, analysis and interpretations.	Link to resource

RELIABILITY AND VALIDITY
Current concepts in validity and reliability for psychometric instruments: theory and application	The authors discuss current models related to reliability and validity, updating prior distinctions of face, content, and criterion validity with the unitary concept "construct validity," and the five sources of evidence to support claims of validity.	Link to resource
Reliability and validity in educational measurement	The authors provide a summary of traditional concepts related to reliability and validity as applied to educational measurement.	Link to resource

REPOSITORIES FOR MEDICAL EDUCATIONAL RESOURCES
MedEdPortal	This searchable repository of peer-reviewed educational resources is maintained by the Association of American Medical Colleges.	Link to resource
A Comparison of Three Health Care Education Collaboration and Publication Portals	The authors compare three medical education portals—MedEdPORTAL, MedEdWorld, and MERLOT— identified as dynamic portals that promote scholarship and the dissemination of resources in health care education.	Link to resource

RESEARCH QUESTIONS
Research questions, hypotheses and objectives	This paper discusses important considerations in the development of a research question and hypothesis and in defining objectives for research.	Link to resource

SURVEY DESIGN
Teaching Academy Survey Question Development	This resource developed by the Teaching Academy discusses how to craft the ideal survey questions.	Link to resource
Developing questionnaires for educational research (AMEE Guide No. 87)	The guide presents a systematic, seven-step process for designing high-quality questionnaires; these steps synthesize multiple survey design techniques and organize them into a cohesive process for questionnaire developers of all levels.	Link to resource
Selecting, designing, and developing your questionnaire	This practical overview outlines how to do questionnaire research that is well designed and managed, starting with the selection and design of questionnaires.	Link to resource
Survey Development Guidance for Medical Education Researchers	The authors present a flowchart outlining the steps necessary to construct valid and reliable surveys.	Link to resource
Guidelines for constructing a survey	This paper provides guidance to help readers understand the appropriate uses of survey instruments as research tools as well as construct and administer a well-designed questionnaire.	Link to resource
Tracing the Steps of Survey Design: A Graduate Medical Education Research Example	Developing a high-quality survey takes time yet the benefits of following a rigorous, systematic approach to survey design far outweigh the drawbacks. The example outlined here demonstrates the authors recommended survey design process.	Link to resource
Avoiding Five Common Pitfalls of Survey Design	A brief summary of common problems encountered in questionnaire design, with examples of solutions to these problems.	Link to resource
Avoiding Four Visual-Design Pitfalls in Survey Development	This is a brief summary of common design and layout issues that have an impact on the readability and data collection in questionnaire research, and examples of how to avoid these issues are provided.	Link to resource

SYSTEMATIC REVIEWS
Five steps to conducting a systematic review	This paper provides a step-by-step explanation—there are just five steps—of the methods behind reviewing, and the quality elements inherent in each step. An example is used to illustrate approaches to each step of the review process.	Link to resource
Systematic reviews and meta-analyses: a step-by-step guide	This web-based resource provides guidance for conducting a systematic review. This step-by-step guide explains the background to common methodologies, what is involved, and how to get started and the steps to completion.	Link to resource

THEORIES OF LEARNING (SEE ALSO: CONCEPTUAL FRAMEWORKS)
Learning Theories Map	This interactive resource links information about the most common theories of learning applied to education, and organizes then according to key concepts, learning paradigms, theorists and scientific discipline.	Link to resource

WRITING A PAPER FOR PUBLICATION
Authorship Contribution and Determination	Recommendations for Best Practices in Authorship	Link to resource
Writing for academia: getting your research into print (AMEE Guide No. 74)	This guide describes strategies for success in writing for publication, including how to choose an educational research topic, define the question and choose the correct design, know the anatomy of a research paper, write each of the sections, optimize the writing before publication, choose a journal, and respond to editors and reviewers.	Link to resource
Twelve tips for getting your manuscript published	The author outlines twelve practical tips on how to navigate the process of getting a manuscript published, which are generalizable to all fields of scholarly writing. He also provides suggestions about the effective use of tables and figures.	Link to resource
Responding to reviewers' comments as part of writing for publication	This paper provides a resource for authors to help them get their work published. The focus is on dealing with, and responding to, the comments of peer-reviewers.	Link to resource
Review criteria for research manuscripts	The AAMC has published a new resource for medical education scholars and faculty as they seek or provide peer review of manuscripts before publication. Authors can use the criteria in this guide to provide direction in writing their own manuscripts.	Link to resource
Scholarship, publication, and career advancement in health professions education (AMEE Guide No. 43)	This guide presents advice about how to prepare and publish health professions education research reports and other forms of scholarship in professional journals and other outlets. It also provides 21 practical suggestions about how to advance a successful and satisfying career in the academic health professions.	Link to resource

WRITING ABOUT CURRICULUM OR EDUCATIONAL INNOVATIONS
Toward better descriptions of innovations	The author proposes that what is most valuable to readers who seek creative solutions to problems is not just reading another person's report of the results of implementing an innovation, but in addition, reading a reflective, analytical, and scholarly treatment. The author provides criteria for writing a paper describing an innovation.	Link to resource
A suggested outline for writing curriculum development journal articles: the IDCRD format	The authors describe a suggested format for organizing curriculum development manuscripts: introduction, development, curriculum, results, and discussion (IDCRD). Detailed descriptions of each section are discussed.	Link to resource