This weeks #OTalk is on the topic of “Occupational Palliative Rehabilitation” and will be hosted by Jennifer Woods of the HIV, Oncology and Palliative care RCOT Specialist Section (@Rcotsshopc).
Author: @Kirstie_OT
#OTalk 30th May – Occupational Therapy and paid work for individuals with learning disabilities.
This weeks #Otalk is on the topic of “What are the considerations for occupational therapists in practice with regards to paid work for individuals with a learning disability?” and will be hosted by Erin Rolfe (@Erinnnnn14).
#OTalk 9th May – Working out loud on the ‘Career Development Framework: Guiding Principles for Occupational Therapy’.
This weeks #Otalk is about the ‘Career Development Framework” being developed by The Royal College of Occupational Therapists with and project lead Dr Stephanie Tempest.
The chat will be co hosted by @TheRCOT and @SetG75.
#OTalk 2nd May 2017 – Developing Outcome Measures.
This week’s #OTalk is on the topic of developing outcome measures and will be hosted by Alison Laver-Fawcett (@alisonlaverfaw) from York St John University in the UK.
Here’s what Alison had to say:
In the early 1990s I embarked on developing and standardising an occupational therapy assessment. It was a daunting prospect as I had little idea of where to start! The process involved my undertaking several interrelated psychometric research studies and became the focus of my doctoral studies. Luckily I had a lot of excellent support, advise and mentorship on my test development journey and finally, around 5 years later, the test was published (the Structured Observational Test of Function, SOTOF, Laver and Powell, 1995). Since then I have been committed to supporting occupational therapists to use and develop standardised occupational therapy assessments and outcome measurement and so I am delighted to have been asked to host this #OTalk on ‘Developing Outcome measures’.
So why is this topic so important? Outcome measures are required to evaluate the effectiveness of occupational therapy services. Commissioners of services expect service evaluation evidence that draws upon the routine application of robust outcome measures. Outcome measurement is driven by both policy and professional standards. The College of Occupational Therapists began its 2013 Position Statement on ‘Occupational therapists’ use of standardised outcome measures’ with the following assertion:
‘The College of Occupational Therapists promotes the use of evidence-based outcome measures to demonstrate the delivery of high quality and effective occupational therapy services and to provide credible and reliable justification for the intervention that is delivered. Without accruing data from such sources the evidence-base to support the value of occupational therapy will fail to grow and the profession will be challenged to produce the robust information that will be essential to support future commissioning of occupational therapy services’ (p1).
There are quite few psychometric terms related to test development so here are a few definitions for students or as a reminder:
Reliability is ‘the extent to which the same measurements of individuals obtained under different conditions yield similar results,’ (Everitt, 2006, p.200).
Inter-rater reliability is the level of agreement between different raters administering the test (Bowers, 2014)
Test-retest reliability is the ‘correlation of scores obtained by the same person on two administrations of the same test and the consistency of this score over time,’ (Laver Fawcett, 2007, p.198).
Validity relates to whether the outcome measure assesses what it proposes to measure.
Content validity is ‘the degree to which the content of an … instrument is an adequate reflection of the construct to be measured’ (Mokkink et al, 2012: 9).
Face validity is the ‘degree to which (the items of) an … instrument indeed looks as though they are an adequate reflection of the construct to be measured’ (Mokkink et al, 2012: 9).
Whether you are an experienced researcher, a clinician or a student please join us on 2nd May for this #OTalk twitter chat and share your ideas and experience. It is never too early in your occupational therapy career to start engaging in the development of outcome measures. On the Occupational Therapy Programme at York St John University students have been collaborating with me to undertake psychometric studies for their final year dissertation (e.g. see Laver-Fawcett et al, 2016). You may have an idea for a new outcome measure, so where do you start with test development? You may be using an unstandardized outcome measure developed in your service and want to know how you would go about standardising it and checking it is a valid and reliable measure. Although you may not be embarking on developing a new outcome measure from scratch, you may still want or need to engage in related research and undertake psychometric studies. For example, you may identify an outcome measure developed in a different country and want to translate it or develop a culturally sensitive version or need to develop norms for your client population. Previous psychometric research may have had limitations that warrant replication of studies, for example existing studies of reliability or validity may have been conducted using small sample sizes and more robust evidence is required. You may want to evaluate the clinical utility of an outcome measure for your area of practice or understand your clients’ experience of undertaking the test.
Suggested talking points and discussion questions to focus our chat:
- If you were looking to develop an outcome measure what would it be and why?
- What factors should occupational therapists consider before deciding to develop an outcome measure?
- If you are using an unstandardized outcome measure, how would you go about standardising this?
- How can we check the reliability of a new or existing measure?
- How do we know if our outcome measure is really measuring what it was developed to measure?
- If occupational therapists are client centred why are there so few face validity studies of occupational therapy outcome measures?
References
Bowers D. (2014) Medical Statistics from Scratch: An Introduction for Health Professionals. 3rd ed. Chichester: Wiley.
College of Occupational Therapists (COT; 2013) Position statement: Occupational therapists use of standardized outcome measures. London: COT. Available from: https://www.cot.co.uk/sites/default/files/position_statements/public/COT-Position-Statement-measuring-outcomes.pdf (accessed 25 April 2017)
Everitt BS. (2006) Medical Statistics from A to Z: A Guide for Clinicians and Medical Students. Cambridge: Cambridge University Press.
Laver, A. J. & Powell, G. E. (1995). The Structured Observational Test of Function (SOTOF). Windsor: NFER-NELSON.
Laver-Fawcett AJ. (2007) Principles of Assessment and Outcome Measurement for Occupational Therapists and Physiotherapists: Theory, Skills and Application. Chichester: Wiley. Note: BAOT/COT members can access an electronic copy of this book at: http://lib.myilibrary.com.cot.idm.oclc.org/ProductDetail.aspx?id=83859
Laver-Fawcett A J, Brain L, Brodie C, Cardy L, Manaton L (2016) The Face Validity and Clinical Utility of the Activity Card Sort – United Kingdom (ACS-UK). British Journal of Occupational Therapy, 79(8) 492–504. doi:10.1177/0308022616629167. Available from: http://journals.sagepub.com/doi/abs/10.1177/0308022616629167 (accessed 25th April 2017).
Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, Bouter LM and de Vet HCW (2012) COSMIN checklist manual. Available at: http://www.cosmin.nl/images/upload/files/COSMIN%20checklist%20manual%20v9.pdf (accessed 25 April 2017).
Some related resources:
College of Occupational Therapists (nd). Resources to help you choose assessments and outcome measures. Available from: https://www.cot.co.uk/cot-library/assessments-and-outcome-measures (accessed 25 April 2017)
College of Occupational Therapists (nd). Developing an assessment tool or outcome measure. Available from: http://www.cot.co.uk/cot-library/developing-assessment-tool-or-outcome-measure (accessed 25 April 2017)
Laver-Fawcett, A J (2014) Routine standardised outcome measurement to evaluate the effectiveness of occupational therapy interventions: essential or optional? Ergoterapeuten, 4, 28-37. Available from: http://www.ergoterapeuten.no/Admin/Public/Download.aspx?file=Files%2fFiles%2fFagartikler%2foutcome.pdf (accessed 25th April 2017).
Laver-Fawcett AJ (2010). The importance of measuring outcomes, including patient reported outcome measures (PROMS). BAOT Lifelong learning event Slide Share. Available at: https://www.slideshare.net/baotcot/the-importance-ofmeasuringoutcomes (accessed 25 April 2017)
Link to a bibliography for Outcomes and Evaluation of Occupational Therapy: https://lsbu.rl.talis.com/lists/ABE8C0F1-30D2-60C4-137E-3F7348999C39/bibliography.html (accessed 25 April 2017)
Post chat transcript
The Numbers
687,039 Impressions
356 Tweets
41 Participants
285 Avg Tweets/Hour
9 Avg Tweets/Participant
#OTalk Participants
#OTalk 25th April – Media Club- Being human on the internet.
This weeks #OTalk is a media club on the topic of being human on the internet by Bill Wong (@BillwongOT).
The link for this week’s media club can be found here. https://www.youtube.com/watch?v=nm0p2NQPB-M
Here’s what Bill had to say…
As a frequent attendee to occupational therapy conferences across the world since I was an OT student in 2009, I am aware that there have been constant discussions and activities on promoting the occupational therapy profession on the Internet. For example, major OT conferences have been tracking how their conference hashtags are doing on Twitter. Another example will be hashtags such as #ot365, #valueofot, #otdistinctvalue, #otphoto, #otmonth, #otweek.
Then, when occupational therapy made appearances on the Ellen show (a popular daytime afternoon show in the US) and Jeopardy (popular quiz show), it has gone viral on social media. Finally, the 2012 Eleanor Slagle lecture by Dr. Karen Jacobs is a notable scholarly speech on promoting occupational therapy. One of the byproducts is that there has been an increasing number of OT programs globally have introduced to their students the idea of Twitter as a professional development tool.
That said, despite all the buzz about promoting our great profession, I feel there have been two items that are not addressed as much. First, although many of us still feel we need to increase public awareness of our profession, it has been a challenge for many OT students and practitioners to constantly promote our profession on social media due to our busy schedules, social media competencies aside. Second, this is something I feel we have not addressed too much in our OT education, is to promote our profession with effectiveness. This represents the perfect transition of this week’s media club material and my reflection of it for the blog.
I started out my OT journey as a wallflower in OT school. I preferred to be in my own world more than in the company of others. Although I connected with majority of my classmates on Facebook, I did primarily for survival reasons, since I was below average for many of my OT classes in OT school. Meanwhile, since I knew OT was my second chance at a career, I also was trying to make network better than my undergrad days (which I did virtually no networking) so that I wouldn’t repeat my struggles of obtaining employment again. At that time, to the surprise of many people who know me now, I actually merely wanted to be a qualified occupational therapist. Although I had ambitions to achieve great things, I actually had not much idea on social media’s role in the profession. Also, not surprisingly, I still was searching for my professional identity as a member of the OT community.
As many people know, summer 2010 was a crucial point of my life personally and professionally. Making a long story short, finding out that I have Asperger’s Syndrome turned out to be a key foundation for my professional identity. After going through the grieving process through the help of some of my OT colleagues, I realized whatever I share about my lived experiences has immense value if I were courageous enough to share regularly. The kicker was that I must pass my placements and become a qualified occupational therapist. However, it was a difficult time for me because in the process of discovering my diagnosis, I walked out of my first hands-on placement. Because of that, I was in a position where I couldn’t afford to fail any more of such placements, as the consequence of walking out under most circumstances is a fail.
Then, when I became a qualified occupational therapist in fall 2012, I have decided to come full circle to support other OT peers while trying to expand my horizons professionally. At that time, I had received some positive feedback from the local OT community in California regarding my courage to share my lived experiences in autism. Also, I started to build my international connections in OT through participation of our great Twitter chat series. Finally, with the great Sarah Bodell as my preceptor for my OTD project, I was introduced to OT 24 Hour Virtual Exchange for the first time. All that added up to my boldness to share my personal insights from an OT perspective on social media.
Like many newly qualified occupational therapists, however, I had bouts of struggles of finding my voice. My primary struggles came from trying to balance out my two primary occupational roles- as an emerging autism self advocate and an occupational therapist. After all, although there were common grounds between the two roles on autism related issues, mastering the appropriate tones of what I say on social media was the most difficult challenge (which is not surprising given my autism diagnosis). The next most difficult challenge was about sharing my views on autism related subjects. I am thankful for my loyal peers in the OT profession for being patient through my trials and errors on finding my voice. Once I have progressed towards mastery, my social media presence exploded, which was part of the reasons why I was selected to do 2 TEDx Talks. Meanwhile, I have also set a continuous CPD goal since 2014 where I will try to master at least 1 new social media platform per year. Of course, I keep on breaking my Twitter impression records at OT conferences. That said, even though I understood the rationale of my social media related CPD goals, I have received mixed reviews from my peers when I mentioned these goals. The reactions I most often heard was “You are way up there with what you already do social media wise. I can’t even keep up with you. I don’t even understand how you do it, let alone you want to continue to innovate without losing your human touch, in spite you are working 40 or more hours a week.”
In researching for this topic, I also found this interesting fact. We all lie somewhere in the technology adaptation spectrum. 2.5% are innovators, 13.5% are early adopters, 34% are early majority, 34% are late majority, and 16% are laggards. (Source: http://www.ondigitalmarketing.com/learn/odm/foundations/5-customer-segments-technology-adoption/)
That said, watching this presentation has prompted me to create these discussion questions for our chat.
- What social media platforms do you have? For each platform, what purposes are you using it for?
- In regards to social media ideas, do you consider yourself an innovator, early adopter, early majority, late majority, or laggard? Why?
- On a scale of 1-10, how important is online presence in having success in the OT profession? Why?
- What qualities are you looking for on an influential OT student or practitioner online?
- On a scale of 1-10, how good are you trying to build rapport with your social media/online followers? Why?
- On a scale of 1-10, how well do you think you utilize your professional social media accounts (whether it’s for CPD or promoting OT)? Why?
Post Chat Information
The Numbers
1,734,327 Impressions
640 Tweets
58 Participants
512 Avg Tweets/Hour
11 Avg Tweets/Participant
#OTalk Participants
OTalk 