Questions Answered: Research in a digital world

On May 7th we hosted a webinar in our How to Do Research and Get Published series. This webinar was centered around the topic ‘Research in a digital world’ with members and researchers from the Center for Advanced Internet Studies College (CAIS) in Bochum, Germany. We were joined by Dr Esther Laufer, Dr Janet Salmons, Prof Dr Edlyne Anugwom, and Azadeh Shamsi, who each provided invaluable insights and discussed how they had conducted digital research in their latest projects.

As we didn’t have time to answer all the excellent questions posed by our live attendees, this blog post, contributed by our panel and our editorial team, hopes to further shed light on some essential topics related to researching in a digital world.

Whether you were an attendee of the webinar looking for answers to a question you posed, or if you are looking to get practical advice from academic experts about digital research, this post is a great place to start.

If you’re interested in watching the recording of this webinar or exploring one of our previous webinars, complete the form at this link and receive immediate access to our full library of webinar recordings.

What is the minimum sample size you would recommend for a quantitative study and a qualitative study? How should we determine the sample size in carrying out digital research?

Janet:  Like many questions about qualitative research, it depends. Sample size depends on the research problem, the scope of the study, and on the length and depth of interviews. Are you interviewing the participant more than once, or collecting additional data from them?

In other words, if you are doing a single short interview, I'd expect to see more interviews than in a study where in-depth interviews were complemented by other data collection, such as observations, narrative or diary entries, etc.

The conventional wisdom about "saturation" asks whether you are getting new kinds of topics or themes, or are participants voicing the same issues you have heard before? In practice that is hard because recruiting and informing participants is no small feat. If you need to recruit more people, that adds time to the process. Or, if you've already recruited people but decide you have enough, you've wasted their time.

In the end, it is all about what you can defend. Can you justify the reasons why you chose the number of interviews you did, and explain why that number was sufficient? You can certainly find research examples of studies with a wide range of cohort sizes. Look at how they have explained their decisions.

Sage Editorial: The appropriate sample size for your qualitative research will be dependent on the methodology you use and the purpose of the research. There is not one set answer for all qualitative methodologies and research questions. One of the guiding principles to determining qualitative data sample size is that of saturation. One hits saturation of data when adding an additional data point (i.e., interview, observation, etc.) would do little to nothing to change your understanding or results.

For more information about how to apply this in your work, we recommend reading the following:

DiStefano, A. S., & Yang, J. S. (2024). Sample Size and Saturation: A Three-phase Method for Ethnographic Research with Multiple Qualitative Data Sources. Field Methods, 36(2), 145-159. 10.1177/1525822X231194515

Could you please tell me a bit more about digital ethnography? What would you recommend reading about it, especially in terms of research ethics?

Janet: Researchers use a variety of approaches to conduct ethnographic research online. Here are blog posts from Sage authors, interviews, and relevant open-access articles:

Sage Editorial: ‘Digital ethnography’ studies online communities (online fanfiction, gamers, etc.) or how particular communities utilize the digital space. For introductory and digestible information on how to “do” digital ethnography, check out this blog post.

Here’s another article we can suggest related to digital ethnography and ethics: 

Lester, J. N. (2020). Going Digital in Ethnography: Navigating the Ethical Tensions and Productive Possibilities. Cultural Studies ↔ Critical Methodologies, 20(5), 414-424. 10.1177/1532708620936995 

What are synchronous and asynchronous focus groups?

Janet: Here are some resources about focus groups: Focus Groups — Sage Research Methods Community

Online focus groups can use text or video. Synchronous groups can meet on a video conference or group chat platform. Asynchronous focus groups can use discussion boards, private online community spaces, or even email.

What kinds of ethical issues do online researchers need to consider? For example, is it ethical to scrape online forums (e.g., Twitter/X, Reddit) conversations without each individual’s consent to be studied? Also, what kinds of IRB difficulties have you faced for online research projects? How did you overcome it?

Janet: Research ethics is a huge topic! This informative post, interview, and article might be useful. See this collection of posts about IRB and research ethics. Also see these guidelines from the Association of Internet Researchers. The professional society for your field (education, psychology, business etc.) may also have discipline-specific guidelines.

Sage Editorial: There are many ethical considerations to keep in mind when doing online research. Some include getting informed consent of participants, maintaining confidentiality (and when is it necessary versus not), data security (also in relation to confidentiality), anonymity, and many more.

We have ethical statements on the majority of our journals, and we are currently updating the Manuscript Submission Guidelines to be more consistent with our reporting. For any journal looking to get included in Clarivate’s ESCI/SSCI we need to ensure that the content contains the correct ethical statements such as COIs, IRB approval, consent, PRISMA, and CONSORT. 

For further discussion of such concerns, we have listed a few articles on ethics and online research methods. 

How to deal with trustworthiness in Online Research? 

Sage Editorial: We all want to ensure that the research we publish meets strict quality guidelines. Authors reputations, and ours, depend on it; so does the reputation of the authors’ research institutions on the global stage.  

Here you will find guidance on policies that help us achieve this, and what we ask of authors to comply with them. Please refer to Sage’s publication ethics and research integrity policy guidelines for more details.  

We also recommend looking at the guidelines provided by The Committee on Publication Ethics (COPE).   

I'd like to hear a little more about the ethical aspects of using data produced for open audiences.

Sage Editorial: With new agencies and funders outlining policy, there is greater push than ever for open research in all its forms – open data being the most obvious one, but also greater transparency with who contributes to that research as well as seeing this research earlier for faster dissemination of knowledge.   

Sage encourages the early dissemination of research through our own preprint platform, Advance (for social science communities), as well as other preprint platforms and SCNs, such as bioRxiV, preprints.org and Academia.edu.  Sage recognizes many benefits of preprints, including speed, collaboration, credit, feedback, open access and inclusivity.

How do we decide on our research design when doing research digitally rather than in person? What factors should be considered in digital research vis-a-vis traditional research?  

Sage Editorial: How you decide to conduct your research will depend on many factors. Your research methods should always make sense for the question you are asking. Also lead with your research question and what would best suit it. Your own constraints as researchers, whether that be time, financing, geography, language, etc., can also impact whether we do research via digital means/online or in person. If your interviewees are scattered across the globe, then it may not be tenable to conduct all interviews in person, thus, using other online means may be the best option. While digital research is different from “traditional” means, many of the considerations are similar. You want to think about how your methods may influence your participants: how comfortable will they be answering questions? how likely are they to respond and respond fully? Think about the factors that you can and cannot control in the virtual environment.  

In your own research what are your findings regarding the scenario where the interviewer makes an effort to be located in the context of the interviewee in order to better understand the point of view and true meaning of the interviewee - is this acknowledged as contributing to quality of research?

Janet: Both insiders and outsiders can conduct excellent research; however, the preparation and need for transparency are different. This open access article might interest you Positionality: The Interplay of Space, Context and Identity (2022). See these posts and open-access articles to learn more: Positionality and Data Collection, Relationally Approaching Fieldwork: Using Indigenous Methodologies as Ethical Practices in the Context of Human Participant Research, and Research questions: Insider/Outsider perspectives.

Sage Editorial: This ties in very much with our ongoing commitment to participatory research as outlined in our publication ethics and research integrity policy guidelines. We also recommend authors follow the Trust Code, a global code of conduct for ethical research partnerships, and recommend authors include a statement in the methods section describing how their work adheres to the Trust Code’s Articles. 

Can one depend on social media for genuine data collection? 

Sage Editorial: Social media is widely used for data collection. It certainly can be used as a reliable source, when done correctly, but there are definitely things to consider. One issue that can pop up is if you can trust a particular respondent that you find via social media. Additional due diligence may need to be done to make sure the respondent is who they say they are (and making sure you don’t have a respondent that is a bot etc.)

How do we ensure the reliability of responses from social media? 

Sage Editorial: Here is an article useful for this question:

Santinele Martino, A., Perrotta, A., & McGillion, B. J. (2024). Who can you trust these days?: Dealing with imposter participants during online recruitment and data collection. Qualitative Research, 0(0). 10.1177/14687941231224591

As algorithms become more prominent in gatekeeping social media, to what extent are we studying social phenomena or are we studying algorithmic phenomena? 

Sage Editorial: This is a big and fascinating question, that will be hard to address with this one answer, so we encourage you to use this response as a starting point for your own research!

How algorithms may impact your research methods and findings will be complicated and dependent on how you are utilizing social media for your research. If you are using webscraping tools to collect data from social media platforms, as long as the data is complete and/or representative, you should still be capturing phenomena of interest, rather than what an algorithm is feeding you. What matters here is that you are thoughtful with the means by which you are collecting your data and understanding how social media algorithms may be influencing the data you can collect and/or the respondents that will engage with you. 

Can we rely on podcasts in our data collection in quality research? 

Sage Editorial: Podcasts can be a source for data collection (as well as methods for conducting interviews). Be sure to understand why the podcast is helpful to your research and to do your own research on the podcast itself. To answer this question more fully, we would want to know how one intends to use the podcast as data, but it is an option.  

When I conduct interviews via apps like Zoom, to me it is still an interview regardless of the means. So, should I name it a digital data collection? 

Sage Editorial: Conducting interviews via the phone or video call/conference apps is certainly an acceptable form for conducting interviews, though it can depend on the research question. What matters is understanding how the medium you choose to conduct your interview through may affect your interviewee. Be up front with this when writing your research methods section and be thoughtful about how using video conferencing apps may impact your interviewees comfort levels. Be sure they are okay with being recorded and respect it if they are not.  

What strategies can I use to get participants to do a survey that was sent online? 

Sage Editorial: Strategies will be highly dependent on who your audience for the survey is. The best thing you can do is make sure that you are targeting the correct people, which is fully dependent on your research question. Some more general tips to increase your response rate includes:  

  • Including incentives – this could be a guaranteed or raffle-style prize for submitting the survey. 

  • Keep it succinct – the longer a survey is, and the more time people will need to spend on it, the less likely participants will take it and complete it fully. Let participants know up front how long the survey should take.  

  • Show them they matter – explain why taking the survey is important and how they are appreciated! 

How do you handle IRB requirements for the Informed Consent process? Electronic signatures? Or are these waived studies, where consents are not required?

Janet:  It depends on the requirements of the institution. Most accept digital signatures. See these posts written by professionals in the IRB/research ethics field:

Done correctly, all gestures and nonverbal cues can also be effectively explored in virtual settings with proper planning. I would like to know the genuine disparity in dynamics between an in-person and digital interview.

Janet: If you want to observe participants’ non-verbal cues using videoconferencing in interviews, make sure to negotiate that access during the recruitment and consent stage. You need to verify that they can be in a quiet space for the interview, with the camera on.

If you want to observe nonverbal cues in extant videos, such as those posted on YouTube or TikTok, see this article, and Facework in Confessional Videos by YouTube Content Creators (2022) and these open access resources.

What tips can you share with students who would like to delve into sensitive topics from an ethical standpoint?

Janet: These resources might interest you about research with vulnerable participants and sensitive topics.

Please could Azadeh indicate what online ethical measures she took? Can you break down the distinction between public vs private for internet research ethics?

Azadeh: For the online research on Twitter or X platform, I focused only on tweets published by public accounts. I did not use any private information and also anonymized the names of commenters in my research.

Azadeh, how did you decide what online spaces to investigate? I have heard that the Persian online space is dominated by poetry blogs. Did you explore the specific context of the Persian online space?

Azadeh: There are many Persian poetry blogs; however, they were not within the scope of my research topic. I chose to focus on Twitter/X partly because it was a popular platform during the “Women, Life, Freedom” protests, both inside Iran and in the diaspora. It’s also fairly straightforward to identify some of the most widely circulated posts and comments using the platform’s public search function (API). Additionally, Twitter has historically been the most political platform in Iran. For example, Facebook was used to mobilize people for demonstrations during the 2009 protests but has lost its popularity since 2016-2017. On Twitter, people write publicly and often anonymize their accounts, allowing them to speak more freely to the public. Instagram has become politicised for the first time since the “Women, Life, Freedom” movement, with many people using it to publish videos and photos of demonstrations or react to the death of Jina and others who were killed later. However, Instagram users, particularly non-influencers, mostly have private accounts, making it difficult to access their content.

How did you engage with the visual aspect of your data using critical discourse analysis? Did you analyse the visual aspects separately from the text in the post or in combination?

Azadeh: I considered visual and textual content in combination and relation to each other, mostly because most of the visual content includes hashtags and captions as well.

You [Azadeh] mentioned that the marginalisation of women in Iran causes possible participants to have biases given that they are being monitored by their government. How were the possible biases identified and avoided?

Azadeh: In my research, I tried to be self-reflective and reflect on my own positionality, my gender, ethnicity, class background and my position as both an insider and an outsider to my research topic. However, I am aware that the selection of content, tweets, and hashtags for analysis was inevitably shaped to some extent by my subjective bias as a researcher. It is also inevitable that the limited selection of respondents for the interviews, based on snowball sampling, does not fully reflect the diversity and breadth of the ‘Woman, Life, Freedom’ movement, or Iranian women’s perspectives on the movement. Collected data, identification, and prioritization of particular discourses in the tweets and interviews were dependent on my judgement as a researcher to a large extent. These are the limitations of my research, influenced by my topic, the context of my research, and my research methodology (which was qualitative), which inherently includes limitations that are not necessarily avoidable.

Are you using your experience as an Iranian woman, conducting data collection in this manner in your research as well, for example as an autoethnography?

Azadeh: Autoethnography was not considered the main research method in my study. However, in my thesis, I tried to reflect on my own experience as an Iranian woman and included the story of my two trips to Iran to conduct my field research.

Could you also do an interpretative phenomenological analysis of the discourse that you gathered?

Azadeh: Interpretative phenomenological analysis was not the methodology that I thought worked for my research topic and the theoretical framework of my research, as my research was focused on political structures and power relations. Even when I was exploring women’s narratives, I was more focused on their collective experiences, traumas, memories and their imaginations, rather than personal and solitary emotions and experiences.

What is the sample size for your research?

Azadeh: I conducted a discourse analysis of 100 textual and visual tweets and 12 interviews.

What ethical issues did you consider in your study?

Azadeh: There were lots of ethical issues that I faced during my research. One of the issues that I faced was my complex positionality in this research both an insider and an outsider. I think my positionality as an Iranian woman with the same experiences, traumas, emotions, hopes and aspirations for the future of my country as millions of other women made me an insider and helped me to build a trust-based relationship and a connection with my interviewees while we were talking about our experiences. At the same time, despite my fieldwork trip to Iran, I did not directly participate in the recent protests, and I did not experience their moments of fear, distress, and anxiety. I left the country during the first days of the “Women, Life, Freedom movement” to continue my education abroad. And there were moments when I felt that these inherent differences influenced my relationship with interviewees and the outcomes of my research.

Azadeh, what is your focus of research? There are many angles that you explained but what is your main focus?

Azadeh: The main focus of my research was on how through emotion sharing, affective publics or counter publics, enabled by the affordances of digital media, shaped Iranian women’s imagination of the future.

What other types of subjugating women are there in Iran, other than imposing Hijabs?

Azadeh: There are numerous discriminations against women in Iran. For instance, based on the law, men and women in Iran face unequal rights when it comes to divorce and decisions about child custody. Additionally, women encounter several discriminatory rules that restrict their ability to make choices about their lives, such as access to job opportunities. Legally, a husband is considered the head of the family, giving him the authority to decide where the family lives and to stop his wife from pursuing education or taking a job. Moreover, Iranian passport law restricts women's freedom to travel. A married woman needs her husband's permission to get a passport and to travel, and her husband can withdraw this permission at any time for any reason.

To watch the recording of this webinar or explore one of our previous webinars, complete the form at this link and receive immediate access to our full library of webinar recordings.

Research, Author ServicesRachel WolfeResearch in a digital world, how to do research and get published, Webinar, Qualitative research, quantitative research