The Hidden Conference Cost of doing Interdisciplinary Work

Hello blog!

Long time no chat. May was entirely lost in the black hole that is the end of the semester and the start of “academic conferencing.” In the past month, I attended the International Communication Association’s conference (ICA 2019; what I would consider the “main” conference of my primary field, Mass Communication) and a workshop at the the North American Chapter of the Association for Computational Linguistics conference (NAACL NLP+CSS 2019). I have a nice break through the remainder of June and July, and then in August I have one more conference (Association in Education for Journalism and Mass Communication, AEJMC 2019).

Which brings me to my topic of the day: the cost of attending conferences to stay up to date on interdisciplinary scholarship.

Realistically, I work in three intersecting fields (four, if you include my computational stuff separately): Mass Communication, Political Science, and Linguistics. Removing a component of the trifecta is not possible; it would mean fundamentally misunderstanding my research agenda.

There are a lot of benefits and problems to doing interdisciplinary research, which many other scholars have spoken on. I love interdisciplinary work, personally, because that’s where all the enjoyable little questions are. And, as valuable as specialization can be, most research questions can be studied in many ways, depending on the department/discipline you end up in. A question about political language may produce different results if studied in Sociology, Psychology, and Political Science. So, to me, the rigorous thing would be to do interdisciplinary research—to be specific in your question, broad in where you look for theory, and concrete in your study’s operationalization and methodology.

But there are substantial professional costs to doing interdisciplinary work. A Google Scholar search of “interdisciplinary research difficulties” will yield more than enough articles to give you a sense of how much the academy has struggled to deal with interdisciplinary scholars (I choose the word “deal” carefully… rarely do I feel as if the academy “supports” interdisciplinary work).

One of those weirdly silent struggles is the cost of attending oh-so-many conferences. In an ideal world, I’d like to submit to conferences for all the fields I participate in (ICA/AEJMC for Mass Comm, LSA for Linguistics, APSA/MPSA for Political Science, NAACL/CoLing for Computational Linguistics). There conferences are important for many reasons. They help you connect with others to find jobs (a super important thing for any graduate student), they expose you to the latest studies and results in the field, and they help you connect with other people who are doing similar work to you.

But each conference can cost a substantial amount of money to attend. Below are the registration cost of the seven conferences I noted above, and a few others:

Conference 2019 Location Regular Reg Student Reg
AEJMC Toronto $ 215 $ 125
APSA Washington D.C. $ 160 $ 125
CoLing Santa Fe $ 715 $ 500
ICA Washington D.C. $ 300* $ 165
IC2S2 Amsterdam 345 € 195 €
ICCSS Amsterdam 450 € 350 €
LSA NYC $ 86 $ 90
NAACL Minneapolis $ 595 $ 295

(* ICA has tiered prices depending on where your institution is located. These are U.S. prices, Tier A.)

For each conference, you also need to account for hotel and airfare, at minimum. The best conferences are the ones that are proximity close (the location of NAACL, in Minneapolis, was a huge reason why I submitted a paper to begin with), but you are typically looking at between 300 and 500 dollars for a round-trip flight to somewhere-in-the-U.S. (aka: Chicago or DC). Conference hotels usually charge between 175 and 250 per night (graduate students bring down the cost substantially by staying with other graduate students). If you are a lucky young scholar like I am, you will have tt professors who will assist with food and drink for a good portion of the trip, but this is obviously not always the case.

All in all, you can be spending somewhere between 500 and 1000 dollars for each conference you attend. This cost increases considerably for non-(U.S. and European) scholars, who have to not only fly in from another country ($$$ international flights anyone?!) but also apply for visas, an increasingly daunting task (most of my conferences are in the U.S., which makes me double-privileged as a scholar in the States).

If you’re a scholar working in two disciplines, that’s twice the conferences you may need to pay for. Or, you’ll have to sacrifice attending certain conferences in one year to attend another. For a young scholar, particularly one doing interdisciplinary research, not attending a conference means missed opportunities to meet people, connect about research, and find future avenues of collaboration.

Given this, we need to start thinking about the conference model, and how that limits young scholars who cannot normally afford to attend so many conferences. Alternative ways to participate, cheaper locations (and cheaper hotels), and having more included in a registration can go a long way.

Yesterday, I was a footnote in history!

Yesterday, I received exciting news! A piece that I had written with Chris Wells for Columbia Journalism Review was cited in the Mueller Report, which was released a day ago.

The piece that we wrote for CJR focused on news organizations that embedded tweets by Internet Research Agency (IRA) handles into their news stories. We’ve increased the number of outlets analyzed since the CJR piece (it was about 40 when we started, but over 100 now), and our finding still holds: a majority of news organizations cited an IRA account in at least one story.

Contrary to popular opinion, these IRA accounts were not sharing “fake news” (as in: false information). Instead, IRA tweets were often quoted for their salient, often hyper-partisan opinions. For example, one tweet advocated for a Heterosexual Pride Day as a way of inciting LGBTQ activists. Another called refugees, “rapefugees”. These accounts would often portray themselves as American people (e.g., @JennAbrams portrayed herself as a “typical” American girl, as shown by research done by my colleague Yiping Xia), or as groups (like @ten_gop, an IRA account pretending to be Tennessee GOP members, and @blacktivist, an IRA account pretending to be BlackLivesMatter organizers).

This has important implications, and speaks to Muller’s earlier indictment of the IRA, which noted that Russia’s campaign goal was “spread[ing] distrust towards the candidates and the political system in general” (p. 6). Ironically, the discovery of the IRA campaign in the summer/fall of 2017 probably fed into this distrust (especially since news organization were as likely to be “duped” as American citizens).

The (underacted) part where we are referenced focuses on this specific issue—journalists embedded these tweets thinking they reflected the opinions of U.S. citizens. This is incredibly problematic, and something that both academics and journalists want to find solutions for. Following our publication in early of 2018, several news organizations reached out to us regarding the specific articles i which they had unintentionally quoted IRA tweets. The research team was particularly excited by these exchanges because it shows that journalists care, and want to avoid doing this in the future.

Tweets about WI Gubernatorial Race Part I: October 28 to Nov 6

Politically, Wisconsin is quite different from my home state of New York. It’s long been considered a purple, or swing, state. For that reason, Wisconsin has often received extra national attention when it comes to local or state-wide politics.

The 2018 Midterm Elections were another example of this, with many citizens around the country tracking Governor Scott Walker’s race against Superintendent Tony Evers. Today, I explore how Twitter talked about this race in the week leading up to Election night (October 28 to Nov 7). This post will focus on the lead-up to the election. Part II will focus on the last few hours of the election (12:30 to 2:30 on November 7, 2018).

(Note: Tweets were collected using the r package rtweets. All datetimes have been converted to CST. For more information about this collection and analysis, please scroll to the bottom)

A broad temporal view: Oct 28 to Nov 6

In the week leading up to the election, there were several noteworthy spikes. We focus on two in particular: November 1 (8-9pm) and November 4 (7pm).

November 1, 2018 from 8:00-9:59 pm

This was the largest spike for Walker in this week (1568 tweets in two hours). Far and away, the most common verb used was variants of “call” (e.g., “called”/”calls”/”calling”). This is because, that day, Governor Walker said that President Obama was "the biggest liar of the world.” This language (employed by non-journalists and journalists alike) was also employed in leads of news stories in Fox News and The Hill).

November 4, 2018 from 7:00-7:59 PM

Although this peak was not as prominent as the others explored here, it is one of the few times that Evers exceeded Walker in references on Twitter.

Many of these tweets appeared to be campaign-oriented tweets about Evers’ support for Wisconsin residents. Unlike the previous spike, there did not seem to be an event aligned with this moment in time. This suggests that this spike was campaign-induced, rather than naturally generated.

A closer look at Election Day

As can be seen in the above image, attention to the Walker/Evers election peaked after 12:00 AM CST, late in the night relative to other well-watched races that day. Votes rolled in minute by minute, with many outlets (including NYT, one of my main trackers) showing a less than 1% margin for several hours.

Methodology

Tweets were collected using Mike Kearney’s rtweets. I began my search at 2:40 AM CST on November 7, 2018, using the search terms “Scott Walker” OR “Tony Evers” OR “#wipolitics” OR “#wielection“. Twitter’s REST API provides an about 1% random sample of tweets. This yielded about 111,000 tweets.

Tweets were annotated for their part-of-speech and dependency using coreNLP. Within the corpus, there were over three million dependencies.

Time Series of IRA Activity on U.S. Social Media Platforms

So I've been toying around with some of the data on other social media platforms, now that much of it has been made publicly available. I'm looking forward to doing a more systematic analysis of the content. In the meantime, however, here are some counts of IRA activities on different social media platforms from 2015 to 2017. 

I was somewhat surprised to see that the time series did not line up as neatly as I thought they would have. Perhaps these strategies are meant to complement each other? This is where a deeper dive into the content or the account would be more useful. For example, perhaps conservative-imitating IRA accounts (e.g., Twitter's @TEN_GOP) responded to different things compared to liberal imitating IRA accounts (e.g., Facebook/Twitter's @Blacktivist group). 

Given the pending lockdown of information regarding this case, it is more important than ever to share and verify this information. It's a shame researcher do not get much access to this kind of data, as scientific rigor should be the minimum standard for analyzing potential foreign influences into American elections. 

Reddit Data Source: [Link]
Facebook Data Source: [Link]

Advertisements purchased by IRA on Facebook

Submissions to Reddit by IRA-controlled accounts

Tweets written by the IRA

Understanding a little more about recent coverage of Korean-U.S. relations through adjective use

Yesterday, U.S. President Trump pulled out of a "highly-anticipated" summit meeting with North Korea's Kim-Jung Un. Given the freshness of this story, it'll take some time collect enough articles to do an anlaysis of this specific incident. But, in the meantime, some interesting results from my analysis of Korean-U.S. relations in American news below.

(Data cleaned and analyzed using R tidytext, quanteda, and OpenNLP. Graphs produced by ggplot2 or MediaCloud.)

Count of articles using the words "Trump" and "North Korea" in top American news media (digital + traditional). Results gathered using MediaCloud archive.

Count of articles using the words "Trump" and "North Korea" in top American news media (digital + traditional). Results gathered using MediaCloud archive.

As we can see above, the majority of the coverage appeared to be between May 7 (when North Korea claimed to have demolished a nuclear test site) and May 21. Using those two weeks as my window, I pulled all articles referencing "Trump" and "North Korea" from four news outlets: CNN (n =96), Fox (n = 114), the New York Times (n = 89) and the Washington Post (208), a total of 507 news stories.

I tagged all the words in the news stories for their part of speech using OpenNLP. I then pulled out all the adjectives, removed duplicates, and screened them for accuracy (OpenNLP has an above 90% accuracy, but the human eye is critical to ensuring quality results). I finally looked at the use of these adjectives in relation to specific actors/parties (mainly North Korea, South Korea, and the United States). Given the effect of political personalization, I consider both the country name and the name of the leader (e.g., "North Korea" OR "Moon Jae-In" OR "President Moon" OR "Moon Jae In") as keywords. I retained the adjective if it appeared within three words of the NK, SK, or US keywords.

Raw counts are presented below (keep in mind the corpus is not perfectly balanced... also, sorry I was too lazy to reorder the charts XD Just so tired and wanted to practice some code):

Most commonly used adjectives related to Trump/U.S.

 

Most commonly used adjectives related to Kim Jung-Un and North Korea

 

Most commonly used adjectives related to President Moon and South Korea

How President Trump used modal verbs in his Syrian air strikes speech

A list of modal verb usage in President Trump's recent speech on the Syrian air strikes. "American" subjects are highlighted.

Modal (auxiliary) verbs (can, could, should, would, must, will, etc.) are an interesting subset of American language, as they are used to indicate "intention" (the way things "ought to" be or "should" be, or an evaluation of what "can" happen). Different modals have different degrees of "modal force."

Here, you'll see that the modal "can" is the most commonly used, often referring to non-American subjects (e.g., "nations of the world" or "our friends"). In the three instances "can" is used in relation to Americans, two are negations ("No amount of American blood..." and "we cannot..."). In the last instance, "can" is used to express a hope, rather than an intention.

By contrast, the other "can" modals are used effectively as threats and evaluations: "The nations of the world can be judged by the friends they keep" or "Increased engagement [...] can ensure that Iran does not profit..."

This distinction is important because it uses our allies and vague "international norms" to express how the world "can" be. Even though the U.S. is certainly an instigator, the modal verb usage implies that we are trying to distance ourselves from taking ownership of these air strikes. We frame this use of force as unavoidable, because of actions taken by other states. The singular use of "would" and "must" in relation to Russia reinforces this further, especially given that they are used so closely together (Russia was supposed to do something, and then they didn't, so they now must do something else).

To make a long story short, we express the necessity of use of force here by saying what we (the US) "cannot" do, and implying what "can" (implicitly "should") happen as a result.

This speech obviously deviates from President Trump's typical language use, but it goes to show the significance of "use of force": even rebels fall in line when they have to justify violence.

(Hoping to do some analysis of news around this air strike tonight as well!)

Interview with Cap Times

This past week, I interviewed with Capital Times in Madison to talk about a recent co-authored study about Russian propaganda in U.S. news media.

I'm glad that the writer, Lisa Speckhard, did a great job capturing my greatest concern with Russian influence and disinformation. We know that the Russians are not going to stop trying to infiltrate U.S. public discourse. They haven't stopped since WWII, and I doubt they ever will.

What we can do is ask ourselves (1) where are they likely to make their way into American political discourse and (2) what can we do [on our end] to stop it.

Journalists are in a special space, as gatekeepers of information, to both prevent and perpetuate Russian propaganda from amplifying. As we learned through this study, this gate is not impervious... especially now that there are so many gates.

In order to keep our public discourse "pure" (that is, not unknowingly manipulated by foreign influences), we need to be self-reflexive, vigilant, and careful. I am continually reminded of this when news organizations reach out to members of our team asking about various articles that have included IRA-linked tweets. We need more news organizations like this and like Slate, who continue to be critical of their journalistic routines. 

New UW Study on Russian Twitter Trolls in U.S. Media

This past week, my research team published a study on news media's use of tweets written by Russia's Internet Research Agency (a copy of the study can be found here).

We also wrote a parallel article with Columbia Journalism Review.

Importantly, we show that Russian tweets conveying stereotypical partisan beliefs were picked up by a variety of mainstream and partisan news outlets. We are particularly critical of news stories that use "strings of tweet" to represent the vox populi (voice of the people). Unlike the more traditional "man on the street" interviews, tweets used in news stories (particularly online ones) are difficult to verify.

However, as shown by the (admittedly shallow) penetration of IRA tweets, it is still important for journalists to verify these Twitter users to the best of their ability. Journalists can do so by corresponding outside of the tweet-o-sphere (e.g., email), trying to look up the user's name on a search engine, or by looking at that user's past social media history.

Digital, partisan news outlets were particularly susceptible to embedded these IRA tweets. Liberal and conservative organizations both used tweets to convey cheap talk (discourse that supports their position or criticizes their opponents'). If the goal of Russian disinformation in the United States was to increase doubt in the news media system and increase polarization in the civil sphere, the amplification of these messages through partisan outlets represent some measure of success.

 

Why does this matter? Aside from natural concerns about deceptive foreign practices to our public sphere, the appearance of these messages across a broad range of news organizations  (bost partisan and traditional, liberal and conservative) shows how little tweets were checked. It highlights a greater problem: our willingness to promote partisan messages to prove a political point, even if they have little to no journalistic value and are not verified.