NSDL CCLI Dissemination

Workshop Wrapup from an NSF Perspective

Brandon Muramatsu — Sun, 21 Feb 2010 23:25:03 +0000

Note: These are the unedited concluding thoughts from NSF participants. They do not necessarily reflect the views of NSF.

We’re hearing the opinions of individual program officers, everything NSF wants in the solicitation.
Misconception about CCLI program–much of what we’ve talked about as missing is in the program, is allowed for the program.
If a project doesn’t have a good dissemination plan, why are they being funded? NSF responds to what comes across the door.
Research into dissemination will get reviewed as educational research, make sure your methodology is strong.
Phase 3 projects that get funded are typically lots of faculty, lots of schools, may not have enough schools. He suggests potential grantees might scale back total request (from $6M maximum) commensurate with numbers and impact.
Think about scalability and sustainability–we’re going to bring in faculty to workshops, give us $N, can’t just keep bringing in more faculty, need to think about things like train-the-trainer, virtual instruction, get rid of travel costs, go beyond just webinars, blogs, etc.–more than broadcast but also interactivity
Help your communities find the outputs from this workshop
July 2010-CCLI PI meeting: He’d like to see workshop participants volunteering to run a workshop at CCLI meeting.

Recommendations for Colleges/Universities, Professional Societies and NSF

Brandon Muramatsu — Sun, 21 Feb 2010 23:21:28 +0000

Note: This is the unedited list of recommendations developed during the workshop for colleges and universities, professional societies and NSF.

Recommendations for NSF

It’s important to clarify what it wants/means with respect to dissemination.
- “Broader Impact” doesn’t really cover the types of things that faculty really should be thinking about and doing
- Change in title for CCLI program will help, but title and RFP language change should be accompanied by a “marketing plan” as well.
- Distribute the message in lay language, not NSF-speak, please
- What is the measurable goal of “good dissemination” that’s included in the proposal
- Dissemination is not just “I did this”, but it’s also diffusion/use
It’s important to be sure reviewers understand the value and need for diffusion, dissemination, sustainability
- Use orientation session of CCLI review panels, to train reviewers with dissemination.
NSF could fund workshops for dissemination
- Model on webinar with local facilitation/face-to-face workshops with proposal writing, broader impact, evaluation.
NSF could encourage proposals from potential adopters to take materials developed by developers and adopt it or adapt it.
- Current phased-plans encourage the developers to ask for funding to do the dissemination, flip this to require the lead institution/PI to be someone that wants to adopt/adapt the innovation.
NSF could encourage people to submit grants to test whether specific innovations could be demonstrated to be cost effective.
It’s important to continue to clarify the message that “commercialization” is “ok”, it’s not a bad word
- Distribute the message in lay language, not NSF-speak, please
It’s important to improve the connection between DUE/EHR and disciplinary directorates grant programs (and progression of development)
On the NSF Website, it would help raise the visibility of educational innovations to include them in Program Highlights, and highlight CCLI projects on NSF foundation-wide or DUE/EHR pages.
After a project has been funded–ask for details of evaluation plan, dissemination plan (similarly to IRB)
NSF could work on developing criteria, in partnership with professional societies, for which individuals could identify exemplary projects

Recommendations for Colleges and Universities

Colleges/universities should consider using/reallocating existing faculty development funding/travel funds to connect faculty to existing communities/dissemination networks.
- Or, part of requirement of funding is to explicity link to dissemination with professional colleagues, or peers at the university.
Colleges/universities could host educational workshops/fairs regionally around college/university.
- E.g., encourage 4-year colleges and univeristies to host workshops with high-schools–for exchange in both directions
- E.g., community colleges
- Link to NSTA to scale up nationally
Departments should assess what they’re doing with learning and teaching, and develop a plan to get there.
- Examine and remove barriers to innovation.
- Set up working group to look at course evaluation instrument, and if the goal is improved teaching and learning–make sure the instrument leads to it and supports.
College and university administration could give faculty release time to participate in community/network building, possibly as part of faculty development activities.
College and university administration could support dissemination within the university–with a peer in the same department, with colleagues in other departments
- Encourage colleges/universities to highlight innovations across department lines
“Candy store” of options that are available at the campus for individual faculty can use (hosted by colleges/universities, or professional organizations?)
- Institutional incentive to use the innovation, to try educational innovations/approaches in use elsewhere in the department or university
Colleges and universities might need to clarify intellectual property rights on the educational innovations under development
Colleges and universities could develop institution-wide graduate student teaching programs using best practices, etc.
Universities could help streamline institutional review board process; to help PIs understand how it applies to the type of research or development a PI is proposing

Recommendations for Professional Societies

Faculty PIs could encourage their professional societies to raise visibility and stature of educational programs
- Concern about “education” section marginalization…it’s too easy to just say the society has an education section.
- Professional societies could help cross-pollinate educational innovations across universities and disciplines
- For professional societies, especially those that are influential in curriculum and assessment in the discipline, they could encourage institutions that educational research is important and valuable
- Work through state government and state agencies, to impact public colleges and universities (examples of including entrepeneurship as a characteristic in tenure and promotion; accept core courses from community colleges during transfer to colleges/universities)
- Professional societies could participate in identifying the learning outcomes and curricula and courses in the disciplines (examples given of what ABET has done in engineering, MAA in mathematics)
  - Could also center on careers–what’s needed for the career
  - Professional societies could endorse certain research-based teaching methods and under what conditions
  - Professional societies could work on standards for faculty development within the discipline–for the faculty and graduate students.
  - Professional societies could highlight funded educational grants in their areas

Revised: What works and under what conditions?

Brandon Muramatsu — Sun, 21 Feb 2010 22:45:27 +0000

Note: This set of recommendations supercedes the previous notes and includes further clarifications from participants. These are unedited notes from the workshop.

What might work is…Infrastructure that makes it both easy and motivational for educational innovation papers and products to be entered in “standard” form, presented in easily searchable “standard” form and peer reviewed like a review publication with access rates, download statistics available for use in CV.
What might work is…Research effectiveness of dissemination funding and publicize results.
Workshop to focus on consumer-focus on training faculty developers to focus on those who might adopt the material, tailor to:
- Full time faculty or not, graduate students
- Relevant to type of institution
- Online/face-to-face
…Create backup materials (e.g., instructors’ guides, additional problems and examples, more than just what is in the software/innovation that the student sees, so the faculty would not have to create additional information)
…Continued support–once you’ve had a workshop, continue interacting with workshop participants, develop/foster a sense of community
…Remember to make materials flexible (adaptable materials, different environments, courses/labs/institution-type, etc.)
…Advisory Board–active during development
…Instantaneous entry–simple starts to using the materials, test things out
…Include administration–don’t focus exclusively on faculty adopters, but also consider department administrators, student advisors, teaching and learning lab staff, information technology staff
…Use professional organizations–to get significant dissemination activity, involve and/or give leadership role through/to professional organizations, filter through professional organizations, where people looking for information might start, understand the cost/resource implications for the professional organization
…Target audiences–select target audiences, and focus on them
Steps to dissemination: Make them aware, make them interested, enable them to use
…Materials with a lot of “pull” (e.g., online homework, real time data, working 3-d print)
…Well designed face-to-face workshops or webinars
- “Fun”: field trips, big name kickoff speech
- Follow-up: 6 month report back, social networking
…Online professional development connected to materials
What might work…are indexing tools to help matchmake between needs and what materials, and support are available
What might work…feedback loops to share experience reports using materials
…Funnel model: Beyond your own class, needs analysis, find indicators of receptivity, target invitations to leaders
…Understanding how potential adaptors think. Plan dissemination to meet them there (work-flow analysis). Start with what will resonate with potential adopters.
…Providing a platform to share (meetings, publications, exploring ideas to test, community)
…Combined face-to-face and online communities (can be done for almost nothing especially for community college faculty, faculty teaching the same subject, meeting every few weeks)
…Designated mentor-mentee (the mentor is the middle man) relationship to help insure support beyond training, the mentor who is not the innovator is a credible aid.
…Credibility of materials–professional society endorsement, award winners, materials from someone in very much the same situation as the potential adopter

What works and under what conditions?

Brandon Muramatsu — Fri, 19 Feb 2010 03:17:02 +0000

Note: These are the unedited notes from workshop discussions. These have been updated with notes clarified through further discussions.

Workshop participants were asked to answer the questions: “What works and under conditions?”.

What might work is…Infrastructure that makes it both easy and motivational for educational innovation papers and products to be entered in “standard” form, presented in easily searchable “standard” form and peer reviewed like a review publication with access rates, download statistics available for use in CV.
What might work is…Research effectiveness of dissemination funding and publicize results.
Workshop to focus on consumer
- Full time faculty or not
- Type of institution
- Online/face-to-face
…Create backup materials (e.g., instructors’ guides)
…Continued support
…Sense of community
…Remember to make materials flexible
…Advisory Board–active during development
Instataneous entry
Include administration
…Use professional organizations
Target audiences
Steps to dissemination: Make them aware, make them interested, enable them to use
…Materials with a lot of “pull” (e.g., online homework, real time data, working 3-d print)
…Well designed face-to-face workshops
- “Fun”: field trips, big name kickoff speech
- Follow-up: 6 month report back, social networking
…Online professional development
…Funnel model: Beyond your own class, needs analysis, find indicators of receptivity, target invitations to leaders
…Understanding how potential adaptors think. Plan to dissemination to meet them there (work-flow analysis). Start with what will resonate with potential adopters.
…Providing a platform to share (meetings, publications, exploring ideas to test, community)
…Combined face-to-face online communities (can be done for almost nothing especially for CC faculty, faculty teaching the same subject, meeting every few weeks
…Designated mentor-mentee (the mentor is the middle man) relationship to help insure support beyond training, the mentor who is not the innovator is a credible aid.
…Credibility of materials.

Day 1 Summary: Jeanne Narum and Bob Beichner

Brandon Muramatsu — Thu, 18 Feb 2010 22:39:03 +0000

Note: These are the unedited notes from workshop discussions.

Jeanne Narum and Robert Beichner were asked to summarize the day’s discussions.

Definitions, approaches
What works
Suggestions/recommendations for NSF
Next step questions to ask

Thoughts:

Use the theories of learing as we develop theories of change
- Dissemination takes longer than the innovation itself takes.
- Recognize that a lot of things do work, what has been demonstrated to work, how do they adapt, what changes are necessary to fit an individual’s needs
- Theory of Positive Deviance, http://www.positivedeviance.org (Marian Zeitlin, Tufts University)
- Capitalize on peer-to-peer networking
- (Jeanne) If deans and presidents knew the cost of lost students (either that leave, or that don’t apply and register), they would care a lot more about trying innovations especially those that
- What can be done to demonstrate the urgency of the need–not just to faculty, but administrators, the public, congresspeople
- How can current techniques of social networking impact everything we’re talking about?

What one thing has worked?:

If there are central repositories, what are the incentives for sending it to those repositories. Have a referreed process, peer review repository. (David McConnell)
(Elaine Seymour)
Workshops, thinking outside of the box, virtual and online
Identifying things that the community really wants, something with lots of pull (David Yaron)
Target leaders, and provide resources (Nancy Paelez)
Meeting has shifted to embrace diffusion as a dispersed responsibility, it’s not just at the PI level. (Elaine Seymour)

Recommendations to NSF:

Name change in program gives an opportunity, make sure you go far enough (?)
“Transforming” is an interesting word, assumes you know what’s there and you understand what educational research has existed
“Broader Impact” doesn’t really cover the types of things that faculty really should be thinking about and doing
We sort of know “what’s out there”, but we don’t know what to do with it! Faculty are poised to do something, but not sure how.
Start with small steps…(refer to how people learn)
Don’t only fund new stuff, but also fund taking materials from location and bringing it to your location
Reinforce to proposal writers that they are agents of change, include this idea
Be sure reviewers understand the value and need for this activity–diffusion, dissemination, sustainability
Commercialization is “ok”, it’s not a bad word.
Connection between DUE/EHR and the disciplinary directorates (Kelly Gull’s comment)
Kudos to central resource project
DUE should get an internal grant from CISE to build and support cyber-enhanced communities of change.
NSF should fund workshops for disseminations–it’s included in CCLI Central Resource
Interactive workshops with proposal writing, broader impact, evaluation–typically had a face-to-face workshop; have transitioned to webinar with local facilitation. –> NSF willing to talk about this with respect to dissemination.
Institutional culture matters.
Use orientation session of CCLI review panels, to train reviewers with dissemination.
Arrange meeting of CCLI PIs at national meetings at professional societies; semi-organized or informal; PIs sharing with each others (so peer-to-peer)
What do we know and what do we need to know? Who are our audience? Who are the partners?
Encourage people to submit grants to test whether specific innovations could be demonstrated to be cost effective. (Previous Pew grant program via Carol Twigg, that showed innovation was showing money.) Economic benefit of innovation. Linda Slakey from NSF was particularly intrigued by this idea.

Three CCLI Faculty Developer Vignettes

Brandon Muramatsu — Thu, 18 Feb 2010 21:35:15 +0000

Three faculty grantees were asked to provide a five minute vignette on their experiences with disseminating one of their educational innovations.

Cathy Skokan, Colorado School of Mines
Michael Gage, University of Rochester
Robert Beichner, North Carolina State University

Cathy Skokan, Colorado School of Mines

Energy Curriculum for Tribal Colleges
Target audience: Faculty in mathematics, science and technology
Developed a series of courses, showed it to two tribal colleges, went to drawing board with tribal colleges, and put together as six course sequence
Tribal Colleges gave a series of limitations: students are different, they don’t learn from just lecturing need lots of hands on, wide range of faculty (but they’re really the same as everywhere else)
Schools can’t absorb more than one or two courses per year, couldn’t expect the colleges to adopt whole 6 course curriculum
Some faculty only use activities, some use whole modules, some use whole curriculum
High school, middle school and elementary school teachers
Success factors:
- Tribal college community is a specific target community
- 1-2 courses per year
- Activities, modules need to understand financial limitations
- Did conferences and journal articles for dissemination

Michael Gage, University of Rochester

WeBWorK project: http://webwork.maa.org/wiki/Main_Page
About 100 schools are using WeBWorK now
Community wiki for interaction, forum, trying video conferences
Moving from University of Rochester to MAA to raise visibility
Support over 10 years from NSF, moving to MAA over next 5 years with “more dissemination”
Specific Goals: Just trying to move homework online, graded automatically (all homework is graded, almost no work for professor) with instant feedback (students find the feedback extremely useful)
Dissemination to:
- Initial dissemination via research contacts.
- Then spread via Rochester’s department chair and told his colleagues about it. Department heads are especially interested in automatic grading
- Smaller universities hear about it “by word of mouth”. Smaller university faculty are more interested in educational aspects.
- Lots of presentations.
- Graduate students have used as students, and now that they’re teaching.

Robert Beichner, North Carolina State University

SCALE-UP Project: http://www.ncsu.edu/PER/scaleup.html
Work together, leverage community as much as you can
Workshop for high school teachers on SCALE-UP, run by Windward School—the community has taken leadership and builds on work
Be opportunistic, lots of connections that interweave relationships

Professional Associations: Creating Pathways for Innovation

Brandon Muramatsu — Thu, 18 Feb 2010 15:43:47 +0000

Professional Associations: Creatinge Pathways for Innovation

NSDL: What they’ve learned about dissemination of innovation

Brandon Muramatsu — Thu, 18 Feb 2010 15:35:31 +0000

NSDL - www.nsdl.org

Kaye Howe from the National Science Digital Library described what they’ve learned about dissemonation of educational innovation.

Must consider the user that we’re “after”, who is the user?
Kaye gave examples of a number of teachers that are teaching in a wide variety of challenging conditions
- New teacher, with a long term substitute position in a low income school district, teaching 3 sections of integrated science, one of earth science, one of chemistry; needs suggestions for hands on activities for chemistry/integrated science (students have had 7 teachers in the last year) that are “easy” and “low budget” that we can attempt.
NSDL is working with Project Tomorrow in California who has data on schools that is helping them understand the environment of teachers and students in K-12.
Make a connection with people that “need us”.
Content is no longer a scarcity…but, just “dumping more and more materials on a teacher is not helpful”.
So, for K-12 materials need to be aligned to state standards, and curated.
Understand the circumstances under which the teacher teaches, and the contexts in which learning takes place.
She asks us to understand which CCLI materials, in what context and what levels of granularity are useful to higher education faculty and K-12 teachers, the users that will be using the materials. We need to adjust what we do understanding who and where the users are.
NSDL has evolved from a digital library to educational services
- Has worked with already established trusted networks
- NSDL uses Pathways who are stewards of disciplinary or level based communities.
- Networks of networks.
- 27 universities, 38 professional societies 28 education organizations, 9 research organizations, 2 public television stations.

Commercial Dissemination: The Role of Publishers in Fostering Innovation in STEM Education

Brandon Muramatsu — Thu, 18 Feb 2010 15:34:30 +0000

Commercial Dissemination: The Role of Publishers in Fostering Innovation in STEM Education

Beyond Dissemination: Sustained Reform in STEM

Brandon Muramatsu — Thu, 18 Feb 2010 14:26:07 +0000

Beyond Dissemination: Sustained Reform in STEM

NSDL CCLI Dissemination

Workshop Wrapup from an NSF Perspective

Recommendations for Colleges/Universities, Professional Societies and NSF

Recommendations for NSF

Recommendations for Colleges and Universities

Recommendations for Professional Societies

Revised: What works and under what conditions?

What works and under what conditions?

Day 1 Summary: Jeanne Narum and Bob Beichner

Thoughts:

What one thing has worked?:

Recommendations to NSF:

Three CCLI Faculty Developer Vignettes

Cathy Skokan, Colorado School of Mines

Michael Gage, University of Rochester

Robert Beichner, North Carolina State University

Professional Associations: Creating Pathways for Innovation

NSDL: What they’ve learned about dissemination of innovation

Commercial Dissemination : The Role of Publishers in Fostering Innovation in STEM Education

Beyond Dissemination: Sustained Reform in STEM

Commercial Dissemination: The Role of Publishers in Fostering Innovation in STEM Education