Scott Morrison hails 'miracle' as Coalition snatches unexpected victory

The Coalition has been re-elected in a shock result in which Labor lost seats in Queensland, Tasmania and NSW and failed to make more than minimal gains nationally.But former prime minister Tony Abbot...

Michelle Grattan, Professorial Fellow, University of Canberra - avatar Michelle Grattan, Professorial Fellow, University of Canberra

Coalition likely to win election in Trump-like upset, but Abbott loses Warringah

Against expectations, Scott Morrison has led the Coalition government back to power.Wes Mountain/The Conversation, CC BY-NDWith 57% of votes counted in the election, the ABC is projecting that the Coa...

Adrian Beaumont, Honorary Associate, School of Mathematics and Statistics, University of Melbourne - avatar Adrian Beaumont, Honorary Associate, School of Mathematics and Statistics, University of Melbourne

Infographic: what we know about the results of Election 2019 so far

As of 10.01pm Saturday, May 18 2019:...

Emil Jeyaratnam, Data + Interactives Editor, The Conversation - avatar Emil Jeyaratnam, Data + Interactives Editor, The Conversation

Bob Hawke, the environmental PM, bequeathed a huge 'what if' on climate change

Since the news broke of his passing, Bob Hawke has been feted as the “environmental prime minister”. From saving the Franklin River, to protecting Antarctica from mining, conservationists ...

Marc Hudson, Researcher, University of Manchester, University of Manchester - avatar Marc Hudson, Researcher, University of Manchester, University of Manchester

You are what you vote: the social and demographic factors that influence your vote

Your income, type of work, where you were born, and other social and demographic factors influences your vote more than you may think.The Conversation / ShutterstockAustralia has changed in many ways ...

Rob J Hyndman, Professor of Statistics, Monash University - avatar Rob J Hyndman, Professor of Statistics, Monash University

View from The Hill: Bob Hawke was master of managing government

It’s always easy to romanticise the past – in celebrating the prime ministership of Bob Hawke it is important to remember it had its peaks and troughs.Trouble marked many years – the...

Michelle Grattan, Professorial Fellow, University of Canberra - avatar Michelle Grattan, Professorial Fellow, University of Canberra

Vic Stockwell’s Puzzle is an unlikely survivor from a different epoch

Sign up to the Beating Around the Bush newsletter here, and suggest a plant we should cover at batb@theconversation.edu.au.On the western side of Mount Bartle Frere, the tallest mountain in Queensland...

Andrew Thornhill, Research botanist at the Botanic Gardens and State Herbarium of South Australia/Environment Institute, University of Adelaide - avatar Andrew Thornhill, Research botanist at the Botanic Gardens and State Herbarium of South Australia/Environment Institute, University of Adelaide

Vital Signs: for the best election predictions, look to the betting markets, not the opinion polls

It turns out that betting markets are quite good predictors, on average.www.shutterstock.comOpinion polls haven’t done too well in some important recent elections.Polls failed to foresee the Bre...

Richard Holden, Professor of Economics, UNSW - avatar Richard Holden, Professor of Economics, UNSW

What I learned from Bob Hawke: economics isn't an end itself. There has to be a social benefit

When I was growing up in Adelaide in the 1970s I wanted to be like Bob Hawke. Other kids generally wanted to be cricket, football or rock stars. I wanted to be a research officer with the Australian C...

Tim Harcourt, J.W. Nevile Fellow in Economics and host of The Airport Economist, UNSW - avatar Tim Harcourt, J.W. Nevile Fellow in Economics and host of The Airport Economist, UNSW

GetUp!'s brand of in-your-face activism is winning elections – and making enemies

GetUp! protesters outside the second leaders' debate in Adelaide earlier this month.David Mariuz/AAPIt can be hard for a political cause to get noticed in a jaded world awash with information, but con...

Mark Rolfe, Honorary associate, School of Social Sciences, UNSW - avatar Mark Rolfe, Honorary associate, School of Social Sciences, UNSW

VIDEO: Michelle Grattan on the passing of Bob Hawke - and the final campaign push

University of Canberra Deputy Vice-Chancellor Leigh Sullivan speaks to Michelle Grattan about the week in politics. They discuss the passing of former Labor prime minister Bob Hawke and his legacy, as...

Michelle Grattan, Professorial Fellow, University of Canberra - avatar Michelle Grattan, Professorial Fellow, University of Canberra

As we face pressing global issues, the pavilions of Venice Biennale are a 21st century anomaly

One of the most powerful images at this year's Venice Biennale is Christoph Büchel's Barca Nostra, 2018-2019, Shipwreck 18th of April 2015. La Biennale di VeneziaThe 58th Venice Biennale of Art o...

Felicity Fenner, Associate Professor at UNSW Art & Design, UNSW - avatar Felicity Fenner, Associate Professor at UNSW Art & Design, UNSW

This is what happens to a baby's body during birth

Delivering a human baby – which has a large, highly developed brain – is risky for mother and baby. jaredandmelanie/flickr , CC BYPregnancy, labour and delivery are incredibly physically ...

Ian Wright, Professor of Paediatrics and Child Health Research, University of Wollongong - avatar Ian Wright, Professor of Paediatrics and Child Health Research, University of Wollongong

Final poll wrap: Race tightens in Ipsos and Dutton just ahead in Dickson, plus many more seat polls

The election campaign is finally coming to an end, with Australians to head to the polls tomorrow.AAP/Bianca de Marchi/Tracey NearmyThe federal election will be held tomorrow. Polls close at 6pm Austr...

Adrian Beaumont, Honorary Associate, School of Mathematics and Statistics, University of Melbourne - avatar Adrian Beaumont, Honorary Associate, School of Mathematics and Statistics, University of Melbourne

imageTons of social media there for the taking… but is it truly representative of real life?Jürgen Pfeffer, CC BY-SA

Behavioral scientists have seized on social media and their massive data sets as a way to quickly and cheaply figure out what people are thinking and doing. But some of those tweets and thumbs ups can be misleading. Researchers must figure out how to make sure their forecasts and analyses actually represent the offline world.

Big Data’s overwhelming appeal

Imagine you’re interested in analyzing society to learn the answers to questions like: how bad is the flu this year? How will people vote in an upcoming election? How do people talk about and cope with diabetes? You could interview people on the street or call them on their phones. That’s what traditional polling firms do – but it takes time and can be quite costly. A promising alternative involves collecting and analyzing social media data – quickly and for free.

Hundreds of millions of people use social media platforms like Facebook and Twitter every day. Individually, they create traces of their activities when they tweet, like and friend each other. Collectively, these users have produced massive, real-time streams of data that offer minute-by-minute updates on social trends – where people are, what people are doing and what they are thinking about. For the last several years, researchers in academia and industry have been developing ways to utilize this flood of data in their investigations and have published thousands of papers drawing on it.

A typical Twitter study could look like the following. Imagine you’re interested in information diffusion after a tragic event. The moment you hear about such an event – for instance, the Boston Marathon bombing – you activate software on your computer that collects in real time Tweets that contain your keywords of interest – maybe Boston in this case. Since there are no Twitter archives available for researchers, you’d utilize Twitter’s data interface and collect all data that come for free. After a couple of hours or days you stop the data collection and start with the analysis.

imageSo much data, there for the taking….Peter Kirkeskov Rasmussen, CC BY-NC-SA

What to watch out for

Not surprisingly, this effort to measure and predict human behavior from social media data is fraught with pitfalls – both obvious and very subtle. For instance, we know that different social media platforms are preferred by different demographic groups. However, most social media studies don’t carefully account for the fact that Twitter is used mostly in cities or that most Pinterest users are upper middle-class and female. This oversight can introduce serious errors into predictions and measurements.

Many of the “individuals” that populate social media platforms are actually accounts managed by public relations companies (think Justin Bieber or Nike) or not even humans at all but automated robots. Because these accounts aren’t portraying anything that even approximates normal human behavior, studies need to remove such accounts before making predictions. However, finding robot accounts can be quite hard.

Another big issue is how the data are collected to be studied. Academic researchers need free – or at least very cheap – access to social media data to perform their studies. Few social media outlets provide this, with Twitter being the exception. Because social media studies tend to be often based on data that are sampled (researchers get about 1% from the free Twitter interface), it’s often the case that what’s available to researchers might not be a representative sample of the overall social media data.

imageSimply collecting billions of data points isn’t enough.Geoff Livingston, CC BY-SA

How to do it better

In order to realize the immense potential of social media-based studies of human populations, research must tackle these kinds of issues head-on. In our recent paper in Science on caveats for social media researchers, we discuss the need to control for bias in all the ways it appears – through platform-specific population makeup, data collection and user sampling. This will involve improvements both in how data is collected and in how data is processed: for example, better methods for identifying non-human accounts on social media are needed.

Ultimately, researchers must be more aware of what is being analyzed when they work with social media data. What data are actually being collected? What systems are actually being studied? What social processes are actually being observed? Through greater awareness of and attention to these questions, the research community will be better able to realize the great promise of social media-based studies.

image

Jürgen Pfeffer receives funding from NSF, DOD.

Derek Ruths receives funding from SSHRC, NSERC, NSF, Public Safety Canada. He consults for Facebook.

Read more http://theconversation.com/studying-society-via-social-media-is-not-so-simple-34631