NSDL CCLI Dissemination

Note: These are unedited notes from the workshop.

Workshop participants were asked to use the #cclidissem hashtag for the workshop. The following tweets were posted throughout the workshop.

• Thinking more about the meeting there is clear need for longterm supported infrastructure projects. NIH does it, wish NSF could
• Brandon Muramatsu: notes from yesterday’s session for NSF CCLI dissemination of innovation workshop http://bit.ly/clRn7d (repeat)
• Elaine Seymour: Need to explicitly discuss the theories of change in projects: If we do x, we expect y, and test against that
• Elaine Seymour: The flawed theory used is if practices are shown to have value then faculty, depts, and institutions will likely adopt
• Elaine Seymour – Need radicalized senior faculty to support and shelter younger faculty as education innovators.
• Elaine Seymour – STEM ed reform has lacked funder initiated, coherent, sustained, and nationally applied strategies.
• Elaine Seymour – Momentum for STEM ed reform has slowed, even stalled.
• Flora McMartin – Most CCLI PIs use project websites as their primary web-based dissemination >often go away shortly after funding ends
• Flora McMartin – 10% of CCLI thought of putting materials in campus DLs/repositories
• Flora McMartin – NSF-CCLI most often selected mode of dissemination is conference proposals >>unsure impact
• Flora McMartin – CCLI PIs report that they are spending 5-10 yrs on their educational innovation (most grants are <3yrs)
• The new program (old CCLI) will emphasize products that are EFFECTIVE and DISSEMINATABLE
• Russ Pimmel – NSF is changing the name of CCLI grants to highlight the changes in the nature of the program
• Russ Pimmel (NSF) need to get PIs to understand evaluation, they don’t know how to communicate with evaluators
• At a workshop discussing NSF CCLI dissemination of innovation http://bit.ly/clRn7dc

Note: This is the unedited list of recommendations developed during the workshop for colleges and universities, professional societies and NSF.

Recommendations for NSF

• It’s important to clarify what it wants/means with respect to dissemination.
  • “Broader Impact” doesn’t really cover the types of things that faculty really should be thinking about and doing
  • Change in title for CCLI program will help, but title and RFP language change should be accompanied by a “marketing plan” as well.
  • Distribute the message in lay language, not NSF-speak, please
  • What is the measurable goal of “good dissemination” that’s included in the proposal
  • Dissemination is not just “I did this”, but it’s also diffusion/use
• It’s important to be sure reviewers understand the value and need for diffusion, dissemination, sustainability
  • Use orientation session of CCLI review panels, to train reviewers with dissemination.
• NSF could fund workshops for dissemination
  • Model on webinar with local facilitation/face-to-face workshops with proposal writing, broader impact, evaluation.
• NSF could encourage proposals from potential adopters to take materials developed by developers and adopt it or adapt it.
  • Current phased-plans encourage the developers to ask for funding to do the dissemination, flip this to require the lead institution/PI to be someone that wants to adopt/adapt the innovation.
• NSF could encourage people to submit grants to test whether specific innovations could be demonstrated to be cost effective.
• It’s important to continue to clarify the message that “commercialization” is “ok”, it’s not a bad word
  • Distribute the message in lay language, not NSF-speak, please
• It’s important to improve the connection between DUE/EHR and disciplinary directorates grant programs (and progression of development)
• On the NSF Website, it would help raise the visibility of educational innovations to include them in Program Highlights, and highlight CCLI projects on NSF foundation-wide or DUE/EHR pages.
• After a project has been funded–ask for details of evaluation plan, dissemination plan (similarly to IRB)
• NSF could work on developing criteria, in partnership with professional societies, for which individuals could identify exemplary projects

Recommendations for Colleges and Universities

• Colleges/universities should consider using/reallocating existing faculty development funding/travel funds to connect faculty to existing communities/dissemination networks.
  • Or, part of requirement of funding is to explicity link to dissemination with professional colleagues, or peers at the university.
• Colleges/universities could host educational workshops/fairs regionally around college/university.
  • E.g., encourage 4-year colleges and univeristies to host workshops with high-schools–for exchange in both directions
  • E.g., community colleges
  • Link to NSTA to scale up nationally
• Departments should assess what they’re doing with learning and teaching, and develop a plan to get there.
  • Examine and remove barriers to innovation.
  • Set up working group to look at course evaluation instrument, and if the goal is improved teaching and learning–make sure the instrument leads to it and supports.
• College and university administration could give faculty release time to participate in community/network building, possibly as part of faculty development activities.
• College and university administration could support dissemination within the university–with a peer in the same department, with colleagues in other departments
  • Encourage colleges/universities to highlight innovations across department lines
• “Candy store” of options that are available at the campus for individual faculty can use (hosted by colleges/universities, or professional organizations?)
  • Institutional incentive to use the innovation, to try educational innovations/approaches in use elsewhere in the department or university
• Colleges and universities might need to clarify intellectual property rights on the educational innovations under development
• Colleges and universities could develop institution-wide graduate student teaching programs using best practices, etc.
• Universities could help streamline institutional review board process; to help PIs understand how it applies to the type of research or development a PI is proposing

Recommendations for Professional Societies

• Faculty PIs could encourage their professional societies to raise visibility and stature of educational programs
  • Concern about “education” section marginalization…it’s too easy to just say the society has an education section.
  • Professional societies could help cross-pollinate educational innovations across universities and disciplines
  • For professional societies, especially those that are influential in curriculum and assessment in the discipline, they could encourage institutions that educational research is important and valuable
  • Work through state government and state agencies, to impact public colleges and universities (examples of including entrepeneurship as a characteristic in tenure and promotion; accept core courses from community colleges during transfer to colleges/universities)
  • Professional societies could participate in identifying the learning outcomes and curricula and courses in the disciplines (examples given of what ABET has done in engineering, MAA in mathematics)
    • Could also center on careers–what’s needed for the career
    • Professional societies could endorse certain research-based teaching methods and under what conditions
    • Professional societies could work on standards for faculty development within the discipline–for the faculty and graduate students.
    • Professional societies could highlight funded educational grants in their areas

Note: This set of recommendations supercedes the previous notes and includes further clarifications from participants. These are unedited notes from the workshop.

• What might work is…Infrastructure that makes it both easy and motivational for educational innovation papers and products to be entered in “standard” form, presented in easily searchable “standard” form and peer reviewed like a review publication with access rates, download statistics available for use in CV.
• What might work is…Research effectiveness of dissemination funding and publicize results.
• Workshop to focus on consumer-focus on training faculty developers to focus on those who might adopt the material, tailor to:
  • Full time faculty or not, graduate students
  • Relevant to type of institution
  • Online/face-to-face
• …Create backup materials (e.g., instructors’ guides, additional problems and examples, more than just what is in the software/innovation that the student sees, so the faculty would not have to create additional information)
• …Continued support–once you’ve had a workshop, continue interacting with workshop participants, develop/foster a sense of community
• …Remember to make materials flexible (adaptable materials, different environments, courses/labs/institution-type, etc.)
• …Advisory Board–active during development
• …Instantaneous entry–simple starts to using the materials, test things out
• …Include administration–don’t focus exclusively on faculty adopters, but also consider department administrators, student advisors, teaching and learning lab staff, information technology staff
• …Use professional organizations–to get significant dissemination activity, involve and/or give leadership role through/to professional organizations, filter through professional organizations, where people looking for information might start, understand the cost/resource implications for the professional organization
• …Target audiences–select target audiences, and focus on them
• Steps to dissemination: Make them aware, make them interested, enable them to use
• …Materials with a lot of “pull” (e.g., online homework, real time data, working 3-d print)
• …Well designed face-to-face workshops or webinars
  • “Fun”: field trips, big name kickoff speech
  • Follow-up: 6 month report back, social networking
• …Online professional development connected to materials
• What might work…are indexing tools to help matchmake between needs and what materials, and support are available
• What might work…feedback loops to share experience reports using materials
• …Funnel model: Beyond your own class, needs analysis, find indicators of receptivity, target invitations to leaders
• …Understanding how potential adaptors think. Plan dissemination to meet them there (work-flow analysis). Start with what will resonate with potential adopters.
• …Providing a platform to share (meetings, publications, exploring ideas to test, community)
• …Combined face-to-face and online communities (can be done for almost nothing especially for community college faculty, faculty teaching the same subject, meeting every few weeks)
• …Designated mentor-mentee (the mentor is the middle man) relationship to help insure support beyond training, the mentor who is not the innovator is a credible aid.
• …Credibility of materials–professional society endorsement, award winners, materials from someone in very much the same situation as the potential adopter

Note: These are the unedited notes from workshop discussions. These have been updated with notes clarified through further discussions.

Continue Reading…

Note: These are the unedited notes from workshop discussions.

Jeanne Narum and Robert Beichner were asked to summarize the day’s discussions.

• Definitions, approaches
• What works
• Suggestions/recommendations for NSF
• Next step questions to ask

Continue Reading…

Three faculty grantees were asked to provide a five minute vignette on their experiences with disseminating one of their educational innovations.

• Cathy Skokan, Colorado School of Mines
• Michael Gage, University of Rochester
• Robert Beichner, North Carolina State University

Continue Reading…

Kaye Howe from the National Science Digital Library described what they’ve learned about dissemonation of educational innovation.

• Must consider the user that we’re “after”, who is the user?
• Kaye gave examples of a number of teachers that are teaching in a wide variety of challenging conditions
  • New teacher, with a long term substitute position in a low income school district, teaching 3 sections of integrated science, one of earth science, one of chemistry; needs suggestions for hands on activities for chemistry/integrated science (students have had 7 teachers in the last year) that are “easy” and “low budget” that we can attempt.
• NSDL is working with Project Tomorrow in California who has data on schools that is helping them understand the environment of teachers and students in K-12.
• Make a connection with people that “need us”.
• Content is no longer a scarcity…but, just “dumping more and more materials on a teacher is not helpful”.
• So, for K-12 materials need to be aligned to state standards, and curated.
• Understand the circumstances under which the teacher teaches, and the contexts in which learning takes place.
• She asks us to understand which CCLI materials, in what context and what levels of granularity are useful to higher education faculty and K-12 teachers, the users that will be using the materials. We need to adjust what we do understanding who and where the users are.
• NSDL has evolved from a digital library to educational services
  • Has worked with already established trusted networks
  • NSDL uses Pathways who are stewards of disciplinary or level based communities.
  • Networks of networks.
  • 27 universities, 38 professional societies 28 education organizations, 9 research organizations, 2 public television stations.