What is Reliability?
Reliability refers to the consistency of measurement. If two different RBTs watch the same behavior, do they get the same data? If not, the measurement is unreliable. For your RBT practice test, think of reliability as "Repeatability."
The primary way we measure reliability in ABA is through Inter-Observer Agreement (IOA). This involves two people watching the same event and comparing their scores. High reliability means the definition is clear and the staff are well-trained.
On the RBT Practice Exam, you will see questions like: "RBT A counts 10 behaviors. RBT B counts 5 behaviors. Is this data reliable?" The answer is No. Low reliability means we cannot trust the data.
Inter-Observer Agreement (IOA)
IOA is the metric of reliability. We want IOA to be high, usually above 80%.
Total Count IOA: (Smaller Count / Larger Count) x 100.
Example: Observer A sees 10. Observer B sees 12.
Math: 10 / 12 = 0.83 = 83% Reliability. Acceptable.
Exact Count IOA: The most strict method. We compare interval by interval. They must agree exactly on every interval to count.
Clinical Applications
Low reliability usually means the Operational Definition is bad. If "Tantrum" isn't defined well, one RBT counts "crying" and the other waits for "kicking." They see different things.
It can also mean Observer Drift. This happens when an RBT unknowingly changes how they measure over time. Maybe they get lazy or stricter. Regular IOA checks fix this drift.
Reliability vs. Accuracy
Data can be reliable but wrong (Inaccurate). If two thermometers both say it is 100 degrees, they are reliable (consistent). But if it is actually 50 degrees outside, they are both inaccurate.
However, you cannot have Accuracy without Reliability. If the tools change every time, you can never be accurate.
RBT Practice Exam Tips for Reliability
Memorize the IOA formula: Smaller / Larger x 100. It is simple math but appears often.
Understand the purpose of IOA: It is to improve the quality of data. It is not to "get the RBT in trouble." It is a quality control mechanism.
Questions about "Observer Drift" are really questions about Reliability. The solution to observer drift is retraining and IOA.
Professional Standards
You should welcome IOA checks. Having a BCBA sit with you and take data is the best way to prove you are doing a good job.
If you and your colleague disagree on data, don't argue or change your numbers to match. Mark down what you saw. Then discuss the definition afterwards. "Why did you count that? I didn't think it met the criteria." This conversation makes the whole team better.
Reliable data is the bedrock of science. Without it, we are just guessing.
Frequently Asked Questions About Reliability
What is the definition of Reliability?
Consistency of measurement. Two independent observers getting the same result for the same event.
What is IOA?
Inter-Observer Agreement. The percentage of agreement between two independent data collectors.
What is a "good" IOA score?
Generally, 80% or higher is considered acceptable for clinical decisions. For research, 90% is often preferred.
How do I calculate Total Count IOA?
Divide the Smaller Count by the Larger Count and multiply by 100.
What is Observer Drift?
When an observer unknowingly alters the way they apply a definition over time. They "drift" away from the original training.
How do you fix Observer Drift?
By conducting regular IOA checks and retraining on the operational definitions.
Can data be Reliable but Inaccurate?
Yes. If two people both consistently count the wrong thing, they are reliable (they agree), but inaccurate (they are wrong).
What causes low reliability?
Usually vague definitions. If the definition is "Being Naughty," no two people will agree. It must be objective.
How often should IOA occur?
Ideally, about 20-30% of sessions should have some form of reliability check, though in practice it varies.
Does IOA require two people?
Yes. You cannot do IOA by yourself. It requires an independent second observer (usually a BCBA or Lead RBT).
Is Reliability on the RBT Exam?
Yes. You need to know that IOA is the fix for reliability errors and be able to do simple IOA math.
What is "Reactivity"?
When the presence of an observer changes the behavior. Reliability checks can sometimes cause reactivity if the student knows they are being watched by two people.