The movement toward what is often described as Web 2.0 is usually understood as a large-scale shift toward a participatory and collaborative version of the web, where users are able to get involved and create content. As things stand we have so far had little opportunity to explore how new forms of power play out in this context of apparent ‘empowerment’ and ‘democratization’. This article suggests that this is a pressing issue that requires urgent attention. To begin to open up this topic this article situates Web 2.0 in the context of the broader transformations that are occurring in new media by drawing on the work of a number of leading writers who, in various ways, consider the implications of software ‘sinking’ into and ‘sorting’ aspects of our everyday lives. The article begins with this broader literature before exploring in detail Scott Lash’s notion of ‘post-hegemonic power’ and more specifically his concept of ‘power through the algorithm’. The piece concludes by discussing how this relates to work on Web 2.0 and how this work might be developed in the future.

Recent studies have enhanced our understanding of digital divides by investigating outcomes of Internet use. We extend this research to analyse positive and negative outcomes of Internet use in the United Kingdom. We apply structural equation modelling to data from a large Internet survey to compare the social structuration of Internet benefits with harms. We find that highly educated users benefit most from using the web. Elderly individuals benefit more than younger ones. Next to demographic characteristics, technology attitudes are the strongest predictors of online benefits. The harms from using the Internet are structured differently, with educated users and those with high levels of privacy concerns being most susceptible to harm. This runs counter to intuitions based on prior digital divide research, where those at the margins should be most at risk. While previous research on digital inequality has only looked at benefits, the inclusion of harms draws a more differentiated picture.

Recently, there has been an upsurge of attention focused on bias and its impact on specialized artificial intelligence (AI) applications. Allegations of racism and sexism have permeated the conversation as stories surface about search engines delivering job postings for well-paying technical jobs to men and not women, or providing arrest mugshots when keywords such as "black teenagers" are entered. Learning algorithms are evolving; they are often created from parsing through large datasets of online information while having truth labels bestowed on them by crowd-sourced masses. These specialized AI algorithms have been liberated from the minds of researchers and startups, and released onto the public. Yet intelligent though they may be, these algorithms maintain some of the same biases that permeate society. They find patterns within datasets that reflect implicit biases and, in so doing, emphasize and reinforce these biases as global truth. This paper describes specific examples of how bias has infused itself into current AI and robotic systems, and how it may affect the future design of such systems. More specifically, we draw attention to how bias may affect the functioning of (1) a robot peacekeeper, (2) a self-driving car, and (3) a medical robot. We conclude with an overview of measures that could be taken to mitigate or halt bias from permeating robotic technology.

As firms are moving towards data-driven decision making, they are facing an emerging problem, namely, algorithmic bias. Accordingly, algorithmic systems can yield socially-biased outcomes, thereby compounding inequalities in the workplace and in society. This paper reviews, summarises, and synthesises the current literature related to algorithmic bias and makes recommendations for future information systems research. Our literature analysis shows that most studies have conceptually discussed the ethical, legal, and design implications of algorithmic bias, whereas only a limited number have empirically examined them. Moreover, the mechanisms through which technology-driven biases translate into decisions and behaviours have been largely overlooked. Based on the reviewed papers and drawing on theories such as the stimulus-organism-response theory and organisational justice theory, we identify and explicate eight important theoretical concepts and develop a research model depicting the relations between those concepts. The model proposes that algorithmic bias can affect fairness perceptions and technology-related behaviours such as machine-generated recommendation acceptance, algorithm appreciation, and system adoption. The model also proposes that contextual dimensions (i.e., individual, task, technology, organisational, and environmental) can influence the perceptual and behavioural manifestations of algorithmic bias. These propositions highlight the significant gap in the literature and provide a roadmap for future studies.

We explore data from a field test of how an algorithm delivered ads promoting job opportunities in the science, technology, engineering and math fields. This ad was explicitly intended to be gender neutral in its delivery. Empirically, however, fewer women saw the ad than men. This happened because younger women are a prized demographic and are more expensive to show ads to. An algorithm that simply optimizes cost-effectiveness in ad delivery will deliver ads that were intended to be gender neutral in an apparently discriminatory way, because of crowding out. We show that this empirical regularity extends to other major digital platforms.

Software applications (apps) are now prevalent in the digital media environment. They are the site of significant sociocultural and economic transformations across many domains, from health and relationships to entertainment and everyday finance. As relatively closed technical systems, apps pose new methodological challenges for sociocultural digital media research. This article describes a method, grounded in a combination of science and technology studies with cultural studies, through which researchers can perform a critical analysis of a given app. The method involves establishing an app’s environment of expected use by identifying and describing its vision, operating model and modes of governance. It then deploys a walkthrough technique to systematically and forensically step through the various stages of app registration and entry, everyday use and discontinuation of use. The walkthrough method establishes a foundational corpus of data upon which can be built a more detailed analysis of an app’s intended purpose, embedded cultural meanings and implied ideal users and uses. The walkthrough also serves as a foundation for further user-centred research that can identify how users resist these arrangements and appropriate app technology for their own purposes.

Information providing and gathering increasingly involve technologies like search engines, which actively shape their epistemic surroundings. Yet, a satisfying account of the epistemic responsibilities associated with them does not exist. We analyze automatically generated search suggestions from the perspective of social epistemology to illustrate how epistemic responsibilities associated with a technology can be derived and assigned. Drawing on our previously developed theoretical framework that connects responsible epistemic behavior to practicability, we address two questions: first, given the different technological possibilities available to searchers, the search technology, and search providers, who should bear which responsibilities? Second, given the technology’s epistemically relevant features and potential harms, how should search terms be autocompleted? Our analysis reveals that epistemic responsibility lies mostly with search providers, which should eliminate three categories of autosuggestions: those that result from organized attacks, those that perpetuate damaging stereotypes, and those that associate negative characteristics with specific individuals.

Health systems rely on commercial prediction algorithms to identify and help patients with complex health needs. We show that a widely used algorithm, typical of this industry-wide approach and affecting millions of patients, exhibits significant racial bias: At a given risk score, Black patients are considerably sicker than White patients, as evidenced by signs of uncontrolled illnesses. Remedying this disparity would increase the percentage of Black patients receiving additional help from 17.7 to 46.5%. The bias arises because the algorithm predicts health care costs rather than illness, but unequal access to care means that we spend less money caring for Black patients than for White patients. Thus, despite health care cost appearing to be an effective proxy for health by some measures of predictive accuracy, large racial biases arise. We suggest that the choice of convenient, seemingly effective proxies for ground truth can be an important source of algorithmic bias in many contexts.Copyright © 2019 The Authors, some rights reserved; exclusive licensee American Association for the Advancement of Science. No claim to original U.S. Government Works.

With the rise of computer algorithms in administrative decision-making, concerns are voiced about their lack of transparency and discretionary space for human decision-makers. However, calls to ‘keep humans in the loop’ may be moot points if we fail to understand how algorithms impact human decision-making and how algorithmic design impacts the practical possibilities for transparency and human discretion. Through a review of recent academic literature, three algorithmic design variables that determine the preconditions for human transparency and discretion and four main sources of variation in ‘human-algorithm interaction’ are identified. The article makes two contributions. First, the existing evidence is analysed and organized to demonstrate that, by working upon behavioural mechanisms of decision-making, the agency of algorithms extends beyond their computer code and can profoundly impact human behaviour and decision-making. Second, a research agenda for studying how computer algorithms affect administrative decision-making is proposed.

Media scholars have studied and critiqued search engines – and in particular the dominant commercial actor, Google – for over a decade. Several conceptual and methodological problems, such as a lack of technological transparency, have made a detailed analysis of concrete power relations and their effects difficult. This paper argues that a microeconomic approach can aid media scholars in examining the complex interactions that underpin the dynamics of information visibility unfolding around the Google search engine. Using the concept of a ‘three-sided market’, we characterize the business model built around google.com as the foundation of the company’s success. We then argue that the combination of search and advertising services, and in particular advertising network services, creates powerful incentives to orient the results page in self-serving ways, leading to fundamental conflicts of interest exacerbated by Google’s dominant position in both markets. Based on search engines’ mass media-like capacity to shape public discourse, we consider the identification of economic forces both as a prerequisite for a robust critique of the current situation and as a starting point for thinking about regulatory measures.

Algorithms, as constitutive elements of online platforms, are increasingly shaping everyday sociability. Developing suitable empirical approaches to render them accountable and to study their social power has become a prominent scholarly concern. This article proposes an approach to examine what an algorithm does, not only to move closer to understanding how it works, but also to investigate broader forms of agency involved. To do this, we examine YouTube’s search results ranking over time in the context of seven sociocultural issues. Through a combination of rank visualizations, computational change metrics and qualitative analysis, we study search ranking as the distributed accomplishment of ‘ranking cultures’. First, we identify three forms of ordering over time – stable, ‘newsy’ and mixed rank morphologies. Second, we observe that rankings cannot be easily linked back to popularity metrics, which highlights the role of platform features such as channel subscriptions in processes of visibility distribution. Third, we find that the contents appearing in the top 20 results are heavily influenced by both issue and platform vernaculars. YouTube-native content, which often thrives on controversy and dissent, systematically beats out mainstream actors in terms of exposure. We close by arguing that ranking cultures are embedded in the meshes of mutually constitutive agencies that frustrate our attempts at causal explanation and are better served by strategies of ‘descriptive assemblage’.

Google’s autocomplete function provides “predictors” to enable quick completion of intended search terms. The predictors reflect the search trends of a population; they capture societal beliefs and perceptions about a variety of subjects. This study explores the predictors provided by Google United States when searching for information about older men and women.

This article examines the extent to which economic, cultural, social, and personal types of engagement with the Internet result in a variety of economic, cultural, social, and personal outcomes. Data from a representative survey of the Dutch population are analyzed to test whether engagement with a certain type of activity is related to “collateral” benefits in different domains of activities, independent from the socioeconomic or sociocultural characteristics of the person. The results show that what people do online and the skills they have affect outcomes in other domains and that this is independent of the characteristics of the person. This means that policy and interventions could potentially overcome digital inequalities in outcomes through skills training and providing opportunities to engage online in a broad variety of ways. A semiologic rather than an economistic approach is more likely to be effective in thinking about and tackling digital inequalities.

Organizations often employ data-driven models to inform decisions that can have a significant impact on people's lives (e.g., university admissions, hiring). In order to protect people's privacy and prevent discrimination, these decision-makers may choose to delete or avoid collecting social category data, like sex and race. In this article, we argue that such censoring can exacerbate discrimination by making biases more difficult to detect. We begin by detailing how computerized decisions can lead to biases in the absence of social category data and in some contexts, may even sustain biases that arise by random chance. We then show how proactively using social category data can help illuminate and combat discriminatory practices, using cases from education and employment that lead to strategies for detecting and preventing discrimination. We conclude that discrimination can occur in any sociotechnical system in which someone decides to use an algorithmic process to inform decision-making, and we offer a set of broader implications for researchers and policymakers.