How To Evaluate the Scientific Research on Dogs and cats: A Guide
Zazie and Kristi chat about the different types of research on dogs and cats, and how pet guardians can interpret it.
This page contains affiliate links which means I may earn a commission on qualifying purchases at no cost to you.
Listen to episode 61 of The Pawsitive Post in Conversation wherever you get your podcasts (Apple, Spotify) or below, watch on Youtube or below, or scroll down for the show notes and transcript.
How to evaluate the scientific research on dogs and cats: A guide
New research on dogs and cats helps us to understand them better and give them better lives. But how do we know if the research is any good, and what about our feelings about the results? Zazie and Kristi chat about how to evaluate the science.
We talk about:
- Taking an ethical stance on dogs and cats that recognizes that they are sentient beings
- How it can be tricky to assess research if you're not trained as a scientist
- The research on training methods shows risks to using aversives and that it's much better to only use reward-based training methods
- Why we look for more than one study to teach us something
- We consider different types of research, including the value of questionnaire studies and of experiments
- What a meta-analysis is
- How it can feel emotional if you don't like the results of a particular study
- People's willingness to take part in research
- A re-evaluation of a classic study on spanking children that is relevant to the discussion
- And finally we talk about the books we're reading
The books we recommend this time are:
- People, Places, Things by Chuck Palahniuk
- New Beginnings at the Cosy Cat Cafe by Julie Haworth
Also mentioned is Zazie's post about a literature review on dog training methods.
Highlights of the conversation on evaluating science on dogs and cats
Zazie: Hey, Kristi. It's great, isn't it, that so many people are interested in learning more about what science tells us about dogs and cats?
Kristi: Absolutely. We love science. We love the scientific process and method. So we're here for it.
Zazie: Yeah. And I love that so many people are interested because so much science is actually very relevant to people's ordinary lives with their dog or with their cat as well. So I think it's really useful.
And it's also really good to talk about how people can evaluate it, because if you yourself are not a scientist,
sometimes it can leave you thinking or with questions about it. And it's good to share some ideas, I think, about how to talk about it. So before we get into this talking about science, though, I just want to say there is also kind of an ethical stance,
and it's worth knowing that there are ethical approaches that are not dependent on research that kind of more reflect how you feel about dogs and cats.
So one of those ethical approaches, and it is backed up by research, but you might have it just as an ethical stance, is the idea that dogs and cats are sentient beings who deserve a good life.
And I think everyone who listens to this podcast will feel that way about their own pets and in the research. That sometimes gets called an animal rights perspective, but that name of it is different from how we might talk about animal rights in another perspective, like in ordinary conversations.
So it's really more about animal welfare. And I think evaluating research is quite tricky if you're not a scientist,
because if you are a scientist, you can go to the paper and you will look very carefully at the method and at the results section to see exactly how they did it and how they did the stats.
And if you're not a scientist. Sometimes that is completely, completely incomprehensible, isn't it?
Kristi: Yeah, I, I think it, it absolutely is. I mean, I think there's a couple of separate issues even with that. Like, number one, the stats. I actually took a. A stats course,
and even though I passed it, I have no ability to understand stats. And whenever I have a question about stats or if I see stats and they're. And. And there's sort of like this assumption that they're saying one thing or the other, I usually go and ask someone who I know is good with that.
So I think, you know, understanding that. Yes, this paper actually says what it's saying. It says is. Is one thing, and the other thing I think is the ability just generally to understand how science is done in any particular discipline.
And I think way back in the Pleistocene, when I did my master's degree,
I did it on animal bone archeology, and I could understand animal bone archeology papers on my specific area, which was a historic site. But that didn't mean that I was comfortable even with animal bone archaeology in a different area or of a different species, like, say, f.
I would not have been able to look at a paper about animal bone archaeology if it was about fish and been competent to even discuss it. So I think there's sort of some expertise needed to sort of unpack the science.
And I think it's.
Yeah, it can be tricky, for sure.
Zazie: Yeah. And I used to teach stats, including multivariate stats, which is the hard. Really quite hard stats too. I used to teach that to unwilling students even when I was a PhD student.
Undergrads had to learn about that.
They didn't really want to know. But I mean, it's hard. And I think even for a research paper that's published, there may be different people on the team who've brought different bits of expertise to the topic.
So there may be one person on the team or two people on the team who've done the stats and other people who've brought other things to it. So that's. That's useful to know.
And I think that means if you're looking at a paper and you're stumped trying to understand something, then, you know, that helps you feel a bit better because, you know, it does take a lot of expertise to do these things. And I've remembered another proviso that I should mention, which is that sometimes these days I see people mention research studies in arguments on social media, and they are studies that do not exist.
Someone has gone to an AI and said, what is the research on this topic that says blah blah. And the AI has made something up, including potentially the names of real people, potentially even the name of a real journal, but more often a kind of made up journal. And I've seen this in discussions where people are citing evidence for their review and I'm like, but that is not a real paper.
So I guess the very first step is to verify is this actually a paper that you can click on and go and read a link?
That's one thing. So if you can go and read the paper, and at least these days it is much easier than ever to get hold of a copy of a paper if you want to.
And also it's useful to know that in most cases, if you email the author of a piece of research, they will usually be happy to send you a copy of it.
Not always. And sometimes it might take them a while to do it because they might be very busy, but they will usually be happy to send you a copy. So you can always get a copy that way as well.
So yeah, I mean research says lots of really useful things about dogs and about cats and about breeds and behavior and training methods and how we interact with them and enrichment and things like that.
So there's a whole lot of different types of science that are interesting here. I don't know which types you're most interested in. Obviously for me is stuff on dog training methods and stuff on enrichment are what I mostly pay attention to.
Kristi: Yeah,
you know, I'm not sure I have like a favorite sub, sub discipline within the dog training research world. I find, you know, I sometimes get these emails from, is it academia? You know, like I get emails about here's a dog paper you might be interested in. And I'm always like, oh well, I hadn't thought about that. That is an interesting topic.
One thing I wish that there was research on is dog. More research on acquired bite inhibition, which I think is something that would be so useful for us as practitioners, but I haven't seen yet.
Zazie: Yeah, that would be really interesting.
So I mean, science doesn't have all the answers for us yet. And I think that can sometimes be one of the tantalizing things when we look at a research study and we think, well that's really interesting.
And now I really, really want to know this. The answer to this other question that they weren't able to look at in the paper because you can't look at everything in one study.
You have to kind narrow it down to something that is manageable that you can do. I mean, you're not writing a book, you're doing one research study and you're not publishing a whole lifetime's work. It's just like one thing.
So scientists have to narrow things down to specific questions that they want to look at.
And the other thing that's important to bear in mind is that the results of one individual study don't necessarily tell us very much. It's when we look at a group of studies that are ideally using different methods and that are pointing to the same kind of result that,
you know, we get really excited and we know that we've got something here which is quite significant.
So even if there's one study that points to something a little bit different, well, then you have to think about why is that different. And that's when looking at the method can be really, really helpful.
Kristi: And I think, yeah, it's just something that pops into my mind is, is when we approach these research studies as we're as dog professionals or as dog guardians, we're not necessarily thinking as a peer reviewer of a paper, you know, but I think it's, it might be useful if we look at these papers because like you said, like the way that the scientific method works,
it's not like they can go, is cat better than dog? You know, it has to be. It has to be.
You know, they're looking at a very specific question that they can address using a very specific method. So I think when we're opening up these papers to sort of set us into the mind of going, hey, how does the scientific method work?
How does science work? And approaching our journey through that paper with that sort of mindset is useful.
Zazie: Yes. And like, if we were thinking, is cat better than dog? Well, in what way is cat better than dog? Is cat better than dog at catching a mouse in the house?
And then, well, which kind of dog? Because are we comparing to my Shih Tzu would not be interested, or are we comparing to a terrier or some other kind of dog which has been bred to catch vermin?
And that's going to be a different kind of question. And how are we going to assess this? And ethically do we care about what's happening to the mice? Are we going to use real mice for this study or do we think, no, actually we don't want the mice to come to any harm.
So we're going to use the scent of mice.
Kristi: We're going to use radio controlled mice.
Zazie: Radio controlled mice with the scent of mice, like mouse urine on Them or something. Can cat detect radio controlled mice better than dog, for example?
Kristi: Let's do this. Send us money and we will do this project.
Zazie: We will have fun getting dogs and cats to play with radio controlled mouse toys and see what happens.
And then maybe another. The question is, well, what do people think about it? And then you're talking about questionnaires as well. So I think one thing I see from some people is that people don't really know how to evaluate questionnaires compared to experiments, and they tend to be asking slightly different questions.
And I think one thing that's really useful to know about questionnaires is that typically someone hasn't just sat down and thought of a load of questions and then just asked them, like, Kristi, do you like cats better than dogs?
Well, that's not a very good question questionnaire.
Typically when people are designing questionnaires, very often actually they're using questionnaires that have been designed and tested and tested again and reused in multiple studies. So they become scales that are very well evaluated.
And we know that you can give it to one person one day and you can give it to them a few weeks later and you're going to get the same result.
And we also know that they're actually measuring something in the real world. So, you know, if I have a question about Kristi, is cat better than dog?
It's actually going to reflect your feelings about Apricat compared to your dogs, for example, as opposed to something else that you might happen, you know, to be thinking of on the day. So questionnaire studies actually are really, really useful and a lot of work goes into designing them.
And that's the other thing. I think if you do a good research study, it can look very simple. It's like any other kind of skill. If you do it well,
may look quite easy, but actually it hides how much work has gone into, you know, getting to that point where you can do it.
Kristi: Yeah, absolutely. And social science is a thing, y'all.
It takes skills and training and education.
Zazie: Yeah, yeah, yeah. So there's a reason people go and spend a long time doing these things.
And then so we have questionnaire studies and questionnaires are really useful. And one of the places, as an example, where we do have a lot of questionnaire studies is of course, in research on dog training methods and the approaches that people use to assess dog training methods.
And that research shows that there are many, many risks to the use of aversive methods. And of course, these questionnaires have been designed in different ways over the years. And just as an example, you can't go to an ordinary pet guardian and say, do you use positive punishment to train your dog?
Because they'll be like, what? I don't know. Know what that is. So you can't use jargon in these. And like, this is one of the things that if you were looking at a study and you're going into the method section, you could look at the specific questions that are used.
Sometimes they're hidden away in supplementary material and you have to click an extra link to see them, but you can look and you can actually see.
And so as an example, one of the ways in which some scientists have dealt with that is that they've written examples of positive punishment and they show them to people and say, do you use these methods?
Or to what extent do you use these methods? On a scale from 1 to 7, for example. And that becomes something that's much easier for people to understand because you have to be speaking the same language as the people that you're studying.
Kristi: Yeah, absolutely. Yeah.
Zazie: I could talk about research methods for a long time and I'm going to make sure we don't spend forever on questionnaires. But I think.
Kristi: I think it's important. I think there's a little bit of. Not aversion to questionnaires, but lack of trust.
Zazie: Yeah. I think a lot of people don't understand how questionnaires are developed. And then there is kind of an important proviso that if you find something from a questionnaire study,
like people who say they use aversive methods also say that their dog is less obedient,
which is the case from multiple different studies, that this isn't necessarily showing a causal link. It's not telling you which direction the causality goes in.
And that's an important thing to bear in mind. And this is where it helps to have other kinds of research and other questionnaires to look at as well.
But I think one thing that we see people say sometimes is that they say correlation doesn't prove causation.
That's true. But then they stop there and they don't think, okay, well, what might be the causation? And it's thinking about that causation that can be really helpful and really useful.
And it can be studies that start with showing correlations that end up proving causation in the end.
And I think the famous example of that would be to do with research on tobacco and cancer.
And for a long, long time, people would say there's an association between smoking and Increased cancer. But it was just in quotes, a correlation. And it took a long time before it was shown that actually there was a causation there.
And of course now we know. So you don't just throw something out and say, oh, it's just a correlation, it's not of interest, it's of huge interest. And you have, you then have to use that information to work out what is actually going on.
Kristi: Yeah, absolutely. And I think like, like you said,
it's not just of interest in how can we figure it out, but it's a, it's, it's of interest in, in just thinking about it. So if there's this correlation between,
you know, disobedience, I'm using air quotes here, and aversives use is then as dog practitioners, if the correlation or if the causation is, you know, dogs who have behavior problems are more likely to experience aversive consequences.
That's something we need to know and we should working on, you know, that we should be as a, as a discipline, as a community be sort of like addressing actively.
Zazie: So yeah, and it gives us also a point to know who we should be helping. Like we should be looking out for people whose dogs have behavior issues because they might be more likely to use aversive methods.
And it seems that those aversive methods, because they have a whole load of risks, they're not necessarily going to be helping the person in that situation anyway or they might be making a problem worse.
And they're not teaching the dog what to do instead and they're not teaching the dog how to cope with a situation that they're finding stressful, for example. So yeah, it's really useful information.
And then especially from medical research, a lot of people really prioritize experimental studies, randomized controlled designs, and they are incredibly useful.
Of course the advantage of questionnaires is they're relating to how ordinary people are doing real things in the real world. Whereas your experiment, your deliberate deliberately taking something probably in your university lab and setting it up to kind of get rid of all these other variables that are going on.
And that has huge advantages and then also some disadvantages as well. So experiments are really helpful and you can set them up to show a causal relationship too.
And when we have all of these different bits of research all pointing in the same direction, that can be really helpful.
And if you're designing your experimental study, for example, well, whatever the topic is, you have to think really carefully about how it's going to work in the lab.
Many of these studies involve Training dogs to do something as well.
If it's cats, then cats don't do so well in the lab. Some cats do. Like, you can bring kittens especially in, and a few adult cats will cope with being put in their carrier and brought into the lab.
But more likely the scientist is going to have to go to the home to meet the cat and do the study there.
And even then the cat might run and hide rather than say, oh, yes, I'd like to take part in an experiment today.
So there are all these other things that can become issues when you're designing an experiment as well. And again, this is why we would want to look at the method and see exactly how someone has done something and have they designed it Right.
So that it's useful, so that the methods used were appropriate for what you're looking at.
And again, I think that's hard to do if you're an ordinary person. That's quite difficult sometimes, but everyone can look with it and have their own opinions on it anyhow.
Which, you know, is always the first place to start.
Kristi: Yeah, absolutely. I, I think,
I think it's. You always learn something. So say, as a member of the public, like me, I always learn something when I'm looking at a method section. Say if, if there was a cat study and, and the researcher was using like citizen science or going to the cat's homes having like reading the,
the method section and, and it's always a surprise to be like, they had to control for this. Oh, isn't that interesting? You know, like you, you wouldn't necessarily have thought that as someone who was like, well, why can't they just tell me why cats do X, Y, Z, you know.
Zazie: Yeah, there's always so much more to think about. So you can look at it. And you're right, we do learn things from how they write about how they've designed it and what they had to do and how they've explained things to people or asked them to do things as well.
And I think citizen science is a nice one. It's. It not so much questionnaires, we don't really tend to call those citizen science so much. It's more when people are doing things.
But all of these things require pet guardians to be willing to take part. And I think that's absolutely wonderful that people are willing and keen to take part. And I see this.
Sometimes researchers ask me to share links to their studies on my page and sometimes they're looking for very specific kinds of people and sometimes they're open to anyone in this country with a dog or anyone in this country with a cat.
And it's always so heartening to see people commenting and saying, I've done it, I've done it, or I've shared it with my friend or whatever. And I think that's really helpful that so many people are willing to take part and help us learn more about dogs and cats.
I think that's wonderful.
Yeah, yeah.
So another thing is sometimes we get studies that are something called a meta analysis.
And what that means is that someone has gone and looked for all the studies on a particular topic,
like all the studies on how dogs interact with children, or all the studies on dog training methods or all the studies on how often people play with their cats and what, if any, behavioral effects there are that kind of thing.
And again, they do this with really specific criteria. So they have to make sure that they've done that search for all the papers really well to begin with.
And then they have specific ways, depending on the type of approach they take, they have specific ways of working through all of that research.
And sometimes they're able to combine things statistically as well and give you an overall stats result that's really helpful. And we have some of those. For example, we have a couple of those on dog training methods,
one of which I've written about on my blog, and that was a study from a while ago now by Gal Ziv. I'll put a link to that in the show notes if anyone wants to go and read about it.
It and these are helpful because they're looking to say, when you look at all the different evidence we've got and you put all of it together, what does it tell us?
And so this is this idea that if lots of things are pointing to the same result, that gives us increased confidence that that result is true and not just an artifact of how the questions were asked or how the study was set up.
I think sometimes when reading studies we read something and we think, oh, but that doesn't apply to me, or oh, that's not what I thought would happen. And so I think sometimes people can quite an emotional response to reading about the results of research as well.
And sometimes we're like, yay,
this confirms what I believe already and we like it. And if it doesn't, then it's a bit more challenging for us, I think, to think about it.
Kristi: Oh, absolutely.
I think sort of like both the meta analysis and the emotional response to a result that doesn't necessarily feel like it fits with our own understanding of our lives or our path.
I think it can be easy for us as guardians to cherry pick a little bit. And I know I'm even guilty of this. I was writing about something that comes to mind as I was writing about dog play at one point and I did a little searchy search on Google Scholar,
found a paper, that's what I wanted to say and just used it.
So I think it can be easy to be like well here's something that says what I want and to use that and having a meta analysis or I mean if we were lucky enough to have textbooks that are of that be like meta, meta kind of analysis and presented in plain language that that is,
can be very useful and I think we can allow ourselves to be sort of open ourselves up emotionally to results that have undergone this kind of meta analysis.
Zazie: Yeah, I think so. And it could be any kind of topic though. I mean there are some topics on which people probably have quite strong emotional feelings anyway.
Maybe some of the things about how people feed their dog and what is best for their dog.
Kristi: Feeding is a big one. That's what I thought of too.
Zazie: Yeah, yeah, that's a big.
Maybe some things about how certain breeds of dog or cat tend to behave or colors of cat like black cats versus tortoiseshell cats versus gray cats or whatever. I think sometimes those are things that we can feel quite emotional about especially if they relate to our own particular pet.
Like if I saw something about tortoiseshell cats, I'm automatically thinking well does this apply to Melina or not? You know, my own cat. And it can feel...
Kristi: Health too. Health of certain breeds I think is a big one too. If you have an admiration for a certain breed and then you read a study about,
you know, how that breed may have health, may struggle with health issues that relate to its breeding I think is something that can be very emotionally fraught.
Zazie: Yeah. And also just really hard as well.
Because we all want to do the very best that we can for our dogs and cats and I think all pet guardians are doing that. And then sometimes some of these studies will show to us that actually maybe something we did in the past wasn't the best thing to do.
And that can be quite hard to deal with.
Yeah, definitely. And you mention stuff to do with confirmation and I think for, I mean I have a flat faced dog,
he's a Shih Tzu, he's very lovely. I got him as an adult rescue. I knew that.
I mean him coming through at that age as a senior dog already, therefore he didn't have the worst examples. And you can see just from looking at his face that he's not got the flattest face of a Shih Tzu out there.
But he does have issues with his eyes now, which he didn't have when we adopted him, but which are very common for the breed. And I think,
you know, I do have feelings about that. And it's hard to see.
He's very good about his eye drops, but it's hard to see that he's suffered from that. So, yeah, we do have.
And I think it's normal to have emotional responses. So just to be clear, because I mentioned meta analyses, when someone does a meta analysis, they have a set of rules to follow.
So they are not cherry picking particular results. They have to show that they have done their search for the papers properly and that they then have evaluated those papers according to a specific framework.
And sometimes they do a narrative analysis of it and sometimes they have other specific approaches that they want to take. But yeah, so I think it's only natural to sometimes have an emotional response.
And sometimes that emotional response is, yay, dogs are amazing, or yay, cats are amazing. And sometimes it's like, oh, I wish I hadn't heard that.
Now I've got to change what I do, or now I've got to think about what I do or whatever. And I think that can be quite challenging.
And of course, no individual study is perfect. For every single study there are always limitations.
And tne of the things I learned quite early when writing my blog is to be really careful about those limitations. Because if you list all of them, people are going to be like, well, this is rubbish.
Why are you telling me about this study? And it might actually have been like a really interesting study.
So I only prefer to pick out the most important limitations and mention those because otherwise everyone is just like,
this is rubbish. And I think it's important to know that there is no perfect study that you can design.
It would be like the most massive, totally impossible to do study if you did that. There are always like swings and roundabouts. This way is better than that way. You have to make choices.
Nothing is perfect. So there always are limitations. And that's another reason why we like to go for everything together, I think.
Kristi: Yeah. And what this is again shining a light on for me is I think, think it's useful to have people like yourself and other science communicators who can sort of ingest these studies and pop them out in a way that,
that respects and honors the, you know, like the science that went into it, but also gives the information to the public in a way that isn't based on an emotional reaction to something like,
you know, like say if, if you feed raw meat to your dog and then you see a study that says, oh, you might be making your partner sick because it, there'll be more of this, this kind of,
you know, exposure, I think going, well, that's not right. I, you know,
instead of listening to someone who is a science communicator or, you know, and sometimes even I think scientists can be good science communicators, but that does not describe all scientists at all.
So, so listening to someone be like, well, here's what it says and here's what this means for you and being able to sort of absorb it and then reflect it out and it is very useful.
Useful.
Zazie: Yeah, I think so, definitely. And I think the scientists who are good at communicating about their research,
I love them, they're amazing. And I really like it when people are like that. But it is a skill and it is not necessarily a skill which is appreciated in academia, depending on where the person is based as well.
So, you know, everyone has their own strengths and weaknesses and I think it's important that other people also are able to look at these studies. And I love writing about science.
I think it's always really fun.
I wanted to share one particular study with you which I read about recently.
And it's not about cats or dogs, it's about children. But it's really interesting and it's really relevant to our conversations about how we treat dogs and cats.
Because recently there was a re evaluation of a much older study,
actually a study from the 80s. So when you look at the research on parenting of children and the disciplinary methods that people use, there is tons and tons of research and a strong concept consensus that corporal punishment, hitting kids, spanking kids is bad for them, and that those effects of it,
they persist into adulthood. So it's not just bad for them in the moment, it kind of has long lasting effects.
But in some of the meta analyses, there's been one specific study mentioned because people have used it to say it shows that spanking works. And so this, I think anyone listening who follows stuff on dog training methods will already be thinking, oh, this sounds kind of familiar.
So this study shows that it works. But more recently, some scientists went back and looked at that specific study.
And the thing is, as things get further and further away,
it's like just this nugget gets taken out and the details of how it was done kind of get forgotten or ignored. And so they went back to the details of how it was done and it was a very specific instance of using punishment to enforce timeout for children.
And it was, it was an experiment. So that's why people have hung onto it is because it's an experiment.
But it was an experiment in which it involved mothers interacting with their kids. And the scientists had earpiece, so they spoke to the mum through an earpiece and told her how to discipline the child according to very specific rules.
So that doesn't at all relate to how people use spanking or punishment in normal life with kids. It was this very specific instance and also although it showed showed that it did successfully enforce a timeout, it wasn't the most effective way of enforcing a timeout.
They tested lots of other different methods and some of those did not involve spanking at all and did also effectively enforce the timeout.
So the result actually, as has been stated in the literature for all of this time, is not quite what people thought. It turns out to be a lot more complex and not actually in favour of using spanking to discipline kids.
And so that's really relevant because many, many countries are signed up to a charter for children. That means that they don't allow spanking of kids. Not all countries are signed up to that.
But I think it's relevant to our discussions to also know that sometimes you can go back to a piece of research and re evaluate it with a modern approach. And the way in which we design experiments has changed as we've, you know, everything's got better.
So sometimes there might be an old study that points in a particular direct, but when you go back and look at it then it's not quite saying what people have thought it was saying over all this time.
So yeah, yeah.
Kristi: As someone who is somewhat familiar with the conversations that go on in dog, in the dog training realm, the fact that hurting or scaring an organism changes their behavior is not surprising at all.
But the fact that it's less effective than positive than other non hurty scary methods to achieve the same goal is also not a surprise at all.
Zazie: So yeah, I think it's really interesting and I love that people are so interested in science. So to everyone listening to this, thank you for your interest in science. We appreciate it too because we love talking about the science.
As an Amazon Associate, I earn from qualifying purchases.
