What is IOA?
Inter-Observer Agreement (IOA) is the degree to which two or more independent observers report the same values after measuring the same events. In ABA, we cannot just "trust" that an RBT is collecting data correctly. We must verify it. IOA is that verification.
For your RBT practice test, IOA is the primary way we assess Reliability of data. It serves two purposes: 1) Quality control of the data, and 2) Performance feedback for the RBT (checking for Observer Drift).
On the RBT Practice Exam, you will likely need to calculate a simple IOA percentage. The standard of excellence in the field is an IOA of 80% or higher.
Common IOA Methods
1. Total Count IOA: The simplest method.
Formula: (Smaller Count / Larger Count) x 100.
Use: For Frequency data. It is crude but easy.
2. Mean Count-Per-Interval IOA: More precise. You calculate agreement for each interval (e.g., Minute 1, Minute 2) and then average them. This prevents "lucky guesses" where the totals match but the timing was off.
3. Trial-by-Trial IOA: Used for Discrete Trial Training (DTT). Did both observers score Trial 1 as "Correct"? Did both score Trial 2 as "Incorrect"?
Formula: (Number of Trials in Agreement / Total Trials) x 100.
Clinical Applications
IOA is mandated by most insurance companies and is a requirement for supervision. Your BCBA will sit with you (or watch remotely) and take data on the same client.
If your IOA is low (e.g., 50%), it usually means the Operational Definition is unclear. If you think "Tantrum" allows crying, but the BCBA thinks "Tantrum" requires kicking, your IOA will be terrible. This triggers a review of the plan.
Observer Drift
Over time, RBTs inadvertently change how they measure. They might become lenient ("I'll give him that point, he tried hard") or strict. This is called Observer Drift. Regular IOA checks detect this drift and correct it back to the standard.
RBT Practice Exam Tips for IOA
Question: "When should IOA be collected?"
Answer: Roughly 20-33% of sessions, across all phases of the study.
Question: "What is the formula for Total Duration IOA?"
Answer: (Shorter Duration / Longer Duration) x 100.
Professionalism
Do not be intimidated by IOA. It is not a test of your character; it is a test of the measurement system. If IOA is low, never change your data to match the BCBA's data. That is falsification.
Report the low IOA honestly. "Hey, we only got 60% agreement. Let's look at the definition of 'Non-compliance' again." This is the sign of a high-quality RBT.
Remember: Independent observation is key. You cannot look at the other person's screen while taking data. That defeats the purpose.
Frequently Asked Questions About IOA
What does IOA stand for?
Inter-Observer Agreement.
What is the minimum acceptable IOA?
Typically 80%. For research purposes, 90% is often preferred.
How do you calculate Total Count IOA?
Divide the Smallest Count by the Largest Count and multiply by 100.
What is the purpose of IOA?
To assess the monitoring reliability of the observers and check for observer drift.
What is Observer Drift?
When an observer unknowingly alters the operational definition over time.
Does IOA prove the data is accurate?
No. It proves the data is reliable (consistent). Two people can agree and still be wrong (inaccurate), though high IOA is a good sign.
What is Trial-by-Trial IOA?
An agreement check used for DTT. You compare the score (+ or -) for each specific trial.
Can I talk to the other observer during IOA?
No. Measurement must be independent. If you talk, you influence each other's data.
How often should IOA occur?
Ideally in about 30% of sessions, covering all experimental conditions.
What is Exact Count IOA?
The most stringent method. The observers must have the exact same count within each specific interval to count as an agreement.
Is this on the RBT exam?
Yes. You will likely see a math question or a conceptual question about why we do it.
Who performs IOA?
Usually the BCBA and the RBT, or two RBTs. It requires two humans observing the same event.