果冻传媒app官方

Table of Contents

No Comment: Public Universities' Social Media Use and the First Amendment

No comment: Social Media Report 2020

This report explains 果冻传媒app官方鈥檚 survey 鈥 covering over 200 public universities and colleges 鈥 concerning how these government institutions filter or block content and users on their official Facebook and Twitter accounts. Using examples of censorship identified in public records acquired from these institutions by 果冻传媒app官方, 鈥淣o Comment: Public Universities鈥 Social Media Use and the First Amendment鈥 discusses how the First Amendment applies to government actors鈥 use of social media and explains why their use of the filtering and blocking tools may violate the First Amendment.


Executive Summary

In October 2019, Facebook founder and chief executive officer Mark Zuckerberg spoke at Georgetown University, extolling the virtues of freedom of expression and noting in particular the importance of college students鈥 ability to 鈥渆xpress who they were and what mattered to them,鈥 including through 鈥渃halleng[ing] some established ways of doing things on campus.鈥[1]

Because Facebook is a private entity, the First Amendment鈥攚hich only limits government actors鈥攄oes not require it to honor expressive freedom. Zuckerberg鈥檚 endorsement of freedom of expression as a principle is a welcome and encouraging development.

It is, however, at odds with the fact that Facebook provides governments the tools to censor. These actors include public universities and colleges which are bound by the First Amendment鈥攖hose very campuses where students have 鈥渃hallenged some established ways of doing things.鈥

These tools include Facebook鈥檚 automated content filters, which allow state institutions to automatically 鈥渉ide鈥 users鈥 comments if they contain words included on Facebook鈥檚 undisclosed list of offensive words or the government actor鈥檚 customized list of prohibited words. These tools enable public universities鈥攁nd other government actors鈥攖o quietly remove critical posts, transforming the Facebook pages into less of a forum and more of a vehicle for positive publicity.

This censorship has deleterious effects on campus and public discourse. Using public records requests, FIREsurveyed over 200 public universities and colleges across 47 states and the District of Columbia. Half of the surveyed institutions choose to use Facebook鈥檚 鈥渟trong鈥 profanity filter, and nearly a third鈥55, or 27.8%鈥攗se the 鈥渕edium鈥 filter. This means that 77.4% of surveyed institutions choose to employ a blacklist of prohibited words not disclosed to the public.

Additionally, nearly a third of the universities surveyed (59, or 30.3%) use a custom blacklist, collectively censoring 1,825 unique words and phrases. These range from familiar profanities to words relating to matters of local and national concern鈥攆or example, blocking animal rights activists鈥 criticism of food vendors, curbing discussion of what to do about a campus Confederate monument, and shielding criticism of itinerant preachers, controversial faculty, politicians, and sports teams.

Even without these tools, public actors can prune their Facebook comments, manually hiding comments to manipulate and shape the apparent public discourse. Wright State University, for example, deleted comments supporting a faculty strike from its Facebook page, leaving behind a community 鈥渇orum鈥 that largely (and falsely) appeared supportive of the university鈥檚 administration and critical of striking faculty.

Because Facebook doesn鈥檛 alert a user when their post has been removed, or tell the public that comments have been censored, these commenters (and other users) may never know their words have vanished.

Further, both Facebook and Twitter allow government actors to block members of the public from participating in these public forums.

These automated methods of censorship are not only contrary to a commitment to freedom of expression, but also provide government actors with tools that鈥攊n light of recent federal court rulings concerning President Trump鈥檚 Twitter feed鈥攙iolate the government actors鈥 legal obligations under the First Amendment.


Methodology

FIRE issued public records requests to five institutions in each state where public institutions of higher education are subject to public records laws, which are similar to the commonly known federal Freedom of Information Act (FOIA).

The five institutions per state were divided into two categories:

  1. The three public, four-year institutions with the largest undergraduate enrollment; and
  2. The two public, two-year institutions with the largest undergraduate enrollment.

Our approach has some logistical limitations. Some states, like Delaware and Pennsylvania, exempt some or all of their institutions of higher education from their public records laws, or limit the scope of the laws to specific expenditures. Other states鈥 public records laws expressly limit their public records laws to requests made by citizens of the state.[2] In other states, there are fewer than five institutions, or fewer than the three four-year and two two-year institutions this survey sought to cover. Finally, some institutions could not be contacted due to the potential for conflict with a pending FIRElawsuit against the institution. In those cases, the institution was replaced with the public institution with the next-highest enrollment. A full list of exceptions may be found in Appendix B.

Altogether, 果冻传媒app官方鈥檚 survey issued public records requests to 224 public universities and colleges in 47 states and the District of Columbia. 198 institutions provided substantive responses. A full list of surveyed institutions may be found in Appendix A, which also identifies the institutions which failed or refused to produce records and includes data compiled from the survey.

Our public records requests identified the official Facebook and Twitter account for the institution and asked the institution to provide to FIREthe Facebook settings, list of blocked Facebook users, and list of blocked Twitter users. An example of a representative request may be found in Appendix C. We encourage journalists or interested members of the public to adapt our request to ask other public institutions鈥攏ot just universities鈥攁bout their records.

The calculation of the number of words on institutions鈥 customized blacklists excludes duplicates. Thus, if two institutions blocked the word 鈥渄og,鈥 that word is counted only once. However, the calculation of the number of blocked Facebook users, as well as the number of blocked Twitter users, did not exclude duplicates. Accordingly, if one user were blocked by two universities, she would be counted twice.


Discussion

Facebook's tools

Facebook allows institutions, entities, organizations, and government actors to establish pages, operating as a forum where the public can share their thoughts with or about the organization. This includes public officials and government entities, ranging from the President of the United States to the City of North Pole, Alaska. As described below, public actors are bound by the First Amendment when they use tools provided by private entities.

When a business, organization, government entity, or public figure creates a 鈥減age,鈥 Facebook allows them to create posts on their own page, gives the option of allowing others to add their own posts, and allows Facebook users to add comments to those posts.

Facebook provides page owners with four tools to limit user content:

  1. The profanity filter. Facebook鈥檚 鈥減rofanity filter鈥 automatically hides 惫颈蝉颈迟辞谤蝉鈥 posts if they contain words on one of two lists鈥攐ne for the 鈥渕edium鈥 setting and one for the 鈥渟trong鈥 setting.[3] The words on these lists are not publicly disclosed, but are composed of 鈥渢he most commonly reported words and phrases marked offensive鈥 by Facebook users.[4] The profanity filter is turned off by default.
  2. The customized blacklist. The 鈥減age moderation鈥 filter allows an administrator to establish a custom list of blocked words.[5] Like the profanity filter, this filter automatically hides posts or comments if they contain a phrase on the custom list.
  3. The blocking function. An administrator of the page may ban particular accounts, after which 鈥渢hey'll no longer be able to publish to [the page], like or comment on your . . . posts, message [you] or like your [page].鈥[6]
  4. Manual removal. An administrator may manually hide specific posts or comments.

A user may not know that their comment has been hidden, as it remains visible to the poster.

Twitter's tools

Twitter offers fewer methods of regulating user interactions with government accounts:

  1. The blocking function. Twitter users can block others from interacting with them.[7]
  2. The muting function. Twitter users can mute users and terms, but this function does not prevent those users from interacting with the account, and a muted user鈥檚 tweets can still be seen by other visitors.[8]

The First Amendment limits public actors鈥 use of filtering tools

With the emergence of the internet and social media, courts have increasingly been called upon to apply the First Amendment to the digital realm. As the Supreme Court of the United States recently observed, 鈥渋n the past there may have been difficulty in identifying the most important places (in a spatial sense) for the exchange of views,鈥 but the answer today is 鈥渃lear鈥: 鈥淚t is cyberspace . . . and social media in particular.鈥[9]

Accordingly, a wide range of courts across the country have held government actors鈥 social media sites,[10] including those on Facebook and Twitter, to be subject to First Amendment limitations.[11] The most prominent of these decisions thus far is a successful challenge to President Trump鈥檚 practice of blocking critics from his Twitter account.[12] These rulings only limit government actors鈥 use of social media tools; the First Amendment does not impose legal obligations on social media sites like Facebook and Twitter, nor on other private parties when they use social media.[13]

What is the public forum doctrine?

The general thrust of the cases addressing public actors鈥 use of social media is that the 鈥渋nteractive space鈥 constitutes a 鈥減ublic forum.鈥 This is a term of art in First Amendment law, setting forth a 鈥渕etaphor . . . first used in constitutional free speech cases as a way of explaining why the government cannot engage in . . . content discrimination with regard to speaking, picketing, or leafleting on city parks and sidewalks.鈥[14]

A public forum is most often seen as a physical site. For example, a public sidewalk or park would likely be seen as a 鈥渢raditional public forum鈥: places where members of the public may freely gather and talk about whatever is on their minds.[15] In those places, attempts to limit viewpoint are never permissible, and attempts to limit content must meet 鈥渟trict scrutiny鈥濃攖hat is, any limits must be necessary to address a compelling government interest and narrowly tailored to serve that interest.

Meanwhile, a meeting space set up by a government entity for discussion of particular subjects or use by particular groups might be a 鈥渓imited public forum.鈥 For example, school board meetings at which members of the public can speak are limited public forums, as they鈥檙e set aside for a particular purpose.[16] In these spaces, government has a freer hand to regulate the subject matter of speech or who may utilize the space, but can鈥檛 exclude people or speech based on the viewpoint expressed. Even if a particular space is privately owned, it is a public forum when a government actor exercises control over it, such as when a city leases a private theater.[17]

Public forum doctrine distinguishes between speech by the government and its employees鈥斺済overnment speech鈥濃攁nd speech by others that the government regulates.[18]

What are the different standards for different public forums?

Not every space opened to expression is a free-for-all in which anything goes. Different standards attach to the forum depending on its purpose, and the extent to which government actors may regulate expression within that forum depends on the forum鈥檚 purpose and the expression at issue:

  • Government actors can almost always limit expression that falls into one of the historic, narrowly-defined exceptions to the First Amendment: 鈥渋ncitement, obscenity, defamation, speech integral to criminal conduct, so-called 鈥榝ighting words,鈥 child pornography, fraud, [and] true threats.鈥[19] This is true in all forums: traditional, designated, and limited.
  • Government actors can never limit expression because of its 鈥減oint of view鈥[20] or where the 鈥渟pecific motivating ideology or the opinion or perspective of the speaker is the rationale for the restriction.鈥[21] This is also true in all forums.
  • If no effort is made to set aside the space for discussion by particular people or about particular subjects, the space is likely to be treated as a designated or 鈥渙pen鈥 public forum.[22] In that case, restrictions on the 鈥渢ime, place, and manner of expression鈥 are permissible only if they are 鈥渃ontent-neutral, are narrowly tailored to serve a significant government interest, and leave open ample alternative channels of communication.鈥[23] If a burden on speech is content-based, the regulation is permissible only if it is 鈥渘ecessary to serve a compelling state interest and that it is narrowly drawn to achieve that end.鈥[24]
  • If a policy is established and consistently enforced[25] to limit use of the forum for particular people or discussion of certain subjects, the space is more likely to be treated as a limited public forum, in which the government has greater leeway to regulate speech.[26] There, restrictions need not be content-neutral, but must be viewpoint-neutral, 鈥渞easonable in light of the purpose served by the forum,鈥[27] and must 鈥渃omport with the definition of the forum鈥[28]鈥攖hat is, the government cannot exclude speech for which, or speakers for whom, the forum was opened.

These standards are also important for evaluating whether the regulation of online expression is permissible under the First Amendment.

How does the public forum doctrine apply to social media?

While public forums are most easily conceptualized as physical spaces鈥攕uch as open areas, meeting rooms, and bulletin boards[29]鈥攖he public forum doctrine has also been applied by the Supreme Court to forums which consist 鈥渕ore in a metaphysical than in a spatial or geographic sense.鈥[30] For example, the same principles have been applied to鈥攁lbeit with varying outcomes鈥攕tudent activity fees pooled for student organizations,[31] schools鈥 internal mailing systems,[32] and email systems.[33] As governments have ventured into cyberspace, courts have applied these principles to government websites, distinguishing government-published websites from websites that allow members of the public to exchange views.[34]

Courts have used the legal framework provided by public forum doctrine to analyze government social media use. While the government actor has control over its own posts (which are quintessential 鈥済overnment speech鈥), the 鈥渋nteractive space鈥 that follows those posts and allows for response from the public constitutes a public forum subject to the First Amendment鈥檚 protection.[35] In other words, while a public university student or faculty member might have a right to post a relevant comment on an existing thread, they could not compel the university to 鈥渟hare鈥 their post in the same manner as the university publishes its own posts.

Here鈥檚 a visual demonstration of how 鈥済overnment speech鈥 and the interactive digital spaces work on Facebook (left) and Twitter (right):

Social Media report 2020: figures 1 and 2

Because the First Amendment protects users鈥 speech in these interactive spaces, government actors like public colleges and universities cannot censor user comments in that space because they disagree with the viewpoint expressed. That means that鈥攁t a minimum鈥攇overnment actors cannot block users,[36] remove posts,[37] or otherwise 鈥渂urden鈥 speech[38] because they find the expression offensive, disagreeable, or wrong.

That does not mean that every digital space associated with a government actor is a free-for-all forum where any content may be shared. How government actors characterize and treat digital spaces is important, as regulations on interactive spaces will be analyzed based on whether they are鈥攁s with physical spaces鈥攖raditional, dedicated, or limited public forums.

If the interactive space is a limited public forum, some restrictions will be permissible. For example, a prohibition on 鈥渃learly off topic鈥 comments is a 鈥渟elf-evidently viewpoint-neutral鈥 regulation 鈥渓imiting a forum to discussion of selected topics,鈥 and is 鈥渞easonably related 鈥 indeed, integral 鈥 to the forum鈥檚 purpose.鈥[39] As such, these kinds of viewpoint-neutral limitations are constitutionally permissible and do not violate the First Amendment.

But the government鈥檚 ability to regulate speech in a limited public forum has boundaries. If, in crafting a policy governing social media content, the policy affords too much discretion to the institution鈥檚 content moderators, those standards will inevitably be abused to censor criticism of the institution. The University of Connecticut鈥檚 policy, for example, reserves to officials 鈥溾 to remove comments they believe to be 鈥,鈥 expression that 鈥,鈥 or posts that are 鈥.鈥

These types of broad, ambiguous policies will lead to viewpoint discrimination and may render the policy itself unconstitutional.[40] For example, Wright State University created 鈥済uidelines鈥 barring comments deemed to be 鈥propaganda,鈥 鈥foul,鈥 鈥trolling,鈥 鈥offensive,鈥 or 鈥渋nflammatory,鈥 giving staff members鈥攐verseen by university administrators鈥攗nfettered discretion to manually hide comments criticizing the administration鈥檚 positions during a faculty strike, such as those disclosing senior administrators鈥 salaries.[41] (At the time of publication, Wright State officials had communicated to FIREthat the university was conducting an internal review of its Facebook guidelines in response to a letter from FIREraising concerns about the policy.)

In addition to the ability to limit expression by policy and manual enforcement, Facebook provides a variety of automated content-policing tools to administrators of Facebook pages.

However, the tools provided by Facebook are not adequately tailored and do not respect the contours of the First Amendment, and their use by government actors will almost certainly lead them to violate the First Amendment rights of students, faculty, and the general public. When used by public institutions, Facebook鈥檚 tools amount to always-on sentinels tasked with identifying and automatically censoring student and faculty speech鈥攁nd, for years, government actors have been using these tools to effectuate online censorship.

Public institutions are using Facebook and Twitter tools to violate the First Amendment

It has long been settled law that the First Amendment is binding on public colleges and universities, including when they act to regulate the otherwise-protected speech of students, faculty, and organizations comprised of students or faculty members, such as student clubs or faculty unions.[42] When public universities open digital spaces for discussion by faculty, students, and the general public, regulations of those digital spaces must meet First Amendment scrutiny.

Beginning in October 2018, FIREissued requests under public records laws to 224 public universities and colleges in 47 states, plus the District of Columbia. The selected institutions represent the three four-year institutions and the two two-year institutions with the highest enrollment in the state.[43]

Of the 224 institutions surveyed, 198 (or 88.4%) provided substantive responses.

FIRE found that many public universities and colleges are using automated tools to regulate online expression in ways that do not comport with the First Amendment.

Most public colleges use Facebook鈥檚 鈥榩rofanity鈥 filters, secret blacklists of words that risk violating the First Amendment.

Half of the responding institutions鈥99 of 198鈥攗se Facebook鈥檚 鈥渟trong鈥 profanity filter, and nearly a third鈥55, or 27.8%鈥攗se the platform鈥檚 medium filter. This means that 77.8% of surveyed institutions use some version of the profanity filter.

Because Facebook does not disclose the list of words or phrases on either the 鈥渕edium鈥 or 鈥渟trong鈥 profanity filters, it is not clear what content is automatically scrubbed from the pages of institutions that utilize the filters created by Facebook. (The custom filters, described below, can be identified through public records requests.)

Facebook, again, does not violate the First Amendment when it creates these filters and offers them to their users. Instead, the First Amendment protects the rights of private entities like Facebook to decide what speech to publish or refuse.[44]

However, when a government agency uses these tools to filter particular words or phrases, it raises serious First Amendment concerns. The premise underlying this type of restriction鈥攖hat blocking individual words is justifiable under the First Amendment because it will improve public discourse鈥攈as been expressly rejected by the Supreme Court. In Cohen v. California, the Court held that a jacket emblazoned with the words 鈥淔uck the Draft,鈥 worn in a courthouse hallway populated with 鈥渨omen and children,鈥 was protected speech.[45] 鈥淸W]e cannot indulge the facile assumption that one can forbid particular words without also running a substantial risk of suppressing ideas in the process,鈥 the Court explained, as 鈥済overnments might soon seize upon the censorship of particular words as a convenient guise for banning the expression of unpopular views.鈥[46]

This risk is most colorfully illustrated by institutions鈥 use of customized blacklists to suppress discussion of local controversies, as discussed below. However, the broader use of Facebook鈥檚 own profanity filters鈥攁 secret list of disapproved words automatically compiled from Facebook users鈥 feedback鈥攚ill have the same effect, while failing to meaningfully protect equitable, civil discourse.[47]

First, because the list is generated by compiling the terms 鈥渕ost commonly reported鈥 as offensive, it necessarily limits speech that is in the greatest need of protection from censorship because it is disapproved by those whose views and sensibilities are more likely to be majoritarian or mainstream. As the Supreme Court observed in holding that burning the American flag is protected expression, 鈥淸i]f there is a bedrock principle underlying the First Amendment, it is that the government may not prohibit the expression of an idea simply because society finds the idea itself offensive or disagreeable.鈥

The use of these databases may also, incidentally, inure to the detriment of minority speakers. One recent study, for example, found that 鈥渢weets by African American authors are 1.5 times more likely to be labeled 鈥榦ffensive,鈥欌 and tweets in African American English are 鈥渕ore than twice as likely to be labeled as 鈥榦ffensive鈥 or 鈥榓busive.鈥欌[48] As a result, automated means of identifying human speech, when relying on human-generated data, may introduce 鈥渉uman biases [which] can easily result in a skewed distribution in the training data,鈥 yielding 鈥渦nintended bias in the resulting models, and therefore potentially unfair applications.鈥[49] Because datasets of offensive language may yield a 鈥渟ystemic racial bias,鈥 they may 鈥渉ave a disproportionate negative impact鈥 on minority communities, effectively creating systems that 鈥渕ay discriminate against the groups who are often the target of the abuse we are trying to detect.鈥[50]

Second, an automated system is unlikely to interpret the context and meaning of uses of words that may be offensive in some contexts and inoffensive in others. Content 鈥渕oderation is, inherently, a subjective practice,鈥 rendering 鈥渃ontent moderation at scale . . . impossible to do well.鈥[51]

As a journalist reporting on recent studies on attempts to create filtering mechanisms for offensive speech observed:

[W]hat is considered offensive depends on social context. Terms that are slurs when used in some settings 鈥 like the 鈥渘-word鈥 or 鈥渜ueer鈥 鈥 may not be in others. But algorithms 鈥 and content moderators who grade the test data that teaches these algorithms how to do their job 鈥 don鈥檛 usually know the context of the comments they鈥檙e reviewing.[52]

Similarly, automated systems are incapable of making value judgments, resulting in the suppression of speech that makes use of offensive words in order to appropriate or report on that speech. While a filter may be intended to prevent, for example, the use of a racial slur directed to another user, it will also suppress posts by a student who, targeted by use of that same slur, posts about their experience on their university鈥檚 Facebook page to call on the university to take action. Activists utilizing offensive language or imagery in order to report on or criticize their use may not anticipate automated filtering, but a user dedicated to using the language or imagery in order to harass others may be more likely to devise workarounds, or use language less likely to be caught in a filter.

Third, the use of blacklists almost certainly fails First Amendment scrutiny under either a designated or limited public forum analysis. Facebook鈥檚 profanity filter is expressly premised on feedback identifying which words or phrases users find offensive. A limitation on this basis is not viewpoint-neutral, rendering its use fatal in any forum.[53] Further, it is unlikely that a public institution will be able to bear its burden鈥攖o demonstrate that its restriction is reasonable鈥攊f it cannot identify the words that it is restricting. Similarly, an undisclosed list of prohibited terms necessarily means that members of the public do not have notice as to what speech is permitted or prohibited, ensuring that the restriction will 鈥渢rap the innocent by not providing fair warning鈥 about what is prohibited.[54]

An automated blacklist also fails First Amendment scrutiny because it automatically hides speech unless it is reviewed by an administrator of the page, effectuating a system of prior restraints on speech, 鈥渢he most serious and the least tolerable infringement on鈥 freedom of expression.[55] The risk prior restraints present to freedom of expression is so great that the 鈥渃hief purpose鈥 in adopting the First Amendment was to prevent their use.[56] Prior restraints are permissible only where they are bound by 鈥渘arrow, objective, and definite standards鈥[57] and where the authorities鈥 review and determination must be made within a specific period of time.[58] Absent these procedural safeguards, which FIREhas not observed at any surveyed institution, a prior restraint is unconstitutional.

An automated system that bars specific words does not differentiate between offensive uses of a word or uses of profound public importance. While it may serve a college or university鈥檚 public relations goals to police public-facing expression by students and faculty for civility, it undermines the institution鈥檚 commitment to freedom of expression and obligations under the First Amendment.

Facebook鈥檚 customizable blacklists are used to restrain speech critical of government institutions and their corporate partners, or speech on matters of public concern.

Nearly a third of the public colleges and universities FIREsurveyed (60, or 30.3%) use a customized blacklist on their Facebook pages, collectively censoring 1,825 unique words and phrases. These range from the words populating George Carlin鈥檚 famous list of 鈥淔ilthy Words鈥[59] to words and phrases pertaining to political and social matters of local and national concern. Together, these customized lists evidence use of Facebook鈥檚 technology to automatically censor criticism of institutions, corporate partners, campus controversies, and even sports-related chest-thumping.

These filters are not likely to pass First Amendment scrutiny. Moreover, many of these filters evidence viewpoint discrimination, as they are specifically designed to hide terms closely associated with particular criticisms. For example, at the University of Kentucky are inarguably intended to suppress criticism by animal rights activists, even if a supporter of the university鈥檚 vendor would also face restrictions because of a post containing the same term.

Even where a filter is not likely to amount to viewpoint discrimination, its use is not likely to be sufficiently tailored to meet the institution鈥檚 obligations under the First Amendment.

For example, a student at the University of North Carolina at Chapel Hill might want to respond to a university post about Black History Month[60] to raise questions about 鈥淪ilent Sam,鈥 a Confederate monument that until recently stood on UNC鈥檚 campus. But they could not have done so if their comment used the monument鈥檚 name, because that phrase was on UNC鈥檚 Facebook blacklist.[61] Or, similarly, when the university posted a message announcing the removal of the monument鈥檚 pedestal,[62] comments containing the words 鈥淪ilent Sam鈥 would unquestionably be relevant. Yet none of the hundreds of visible comments contain those words鈥攁lmost certainly because of the blacklist. Similarly, students who used the phrases 鈥9/11鈥 or 鈥渢errorism鈥 in responding to the university鈥檚 post memorializing the September 11 attacks[63] would not reach their intended audience鈥攁gain, almost certainly because of the blacklist. Yet each of these comments would be on-topic and not otherwise fall within a categorical exception to the First Amendment.

Similarly, students at schools like Oklahoma State University, where the names of political candidates are automatically scrubbed, would encounter difficulty if they mentioned their favored (or disfavored) candidates鈥 names in response to the schools鈥 posts about getting out the vote.[64]

These restrictions, even where they are viewpoint neutral, are not 鈥渞easonable in light of the purpose served by the forum鈥[65] because they burden relevant, on-topic expression in a medium intended to operate in real-time. Instead, they effectuate a system of prior review on particular terms, and institutions are not likely to provide the procedural protections required by the First Amendment.

FIRE鈥檚 survey revealed that public institutions are using the customized blacklist feature to limit a wide range of criticism, or to blunt discussion of local controversies:

  • The University of Kentucky blocks the words 鈥,鈥 鈥,鈥 鈥,鈥 and 鈥.鈥 The university explained to FIREthat it instituted this restriction 鈥渁round the time that Aramark came on campus鈥 and activists began posting 鈥渉ighly graphic videos of chicken slaughter.鈥 In 2014, the university announced it was entering into a 15-year contract worth $250 million with Aramark.[66] Similarly, Mississippi State University blocks mentions of 鈥溾 and .
  • Texas A&M University blocked terms in an effort to bar criticism by animal rights activists, including People for Ethical Treatment of Animals, over research conducted on dogs. These terms include 鈥,鈥 鈥,鈥 鈥,鈥 and 鈥.鈥 PETA and the Electronic Frontier Foundation sued Texas A&M in 2018 over the censorship.[67] Santa Monica College likewise bars 鈥,鈥 鈥,鈥 鈥,鈥 and 鈥溾 following a PETA campaign critical of cat dissection in an anatomy course.[68]
  • During protests over the 鈥淪ilent Sam鈥 Confederate monument, the University of North Carolina at Chapel Hill blocked posts containing the phrase 鈥,鈥 as well as mentions of 鈥.鈥 The university also blocked mentions of the terms 鈥,鈥 鈥,鈥 鈥,鈥 and the name of a professor whose teaching about the September 11, 2001, terrorist attacks drew national media coverage when a student, who had not read any of the materials,[69] publicly complained about the materials read in the course.[70] The university also barred .
  • The University of Arizona automatically removes posts or the name of an itinerant preacher known for holding signs reading 鈥淵ou Deserve Rape,鈥 which has drawn media criticism.[71] This restriction presumably removes complaints about the preacher.
  • Suffolk County Community College (NY) blocks posts concerning inclement weather closures, and includes terms that demonstrate that the purpose is to inhibit criticism, barring the words 鈥,鈥 鈥,鈥 鈥,鈥 鈥,鈥 鈥,鈥 鈥,鈥 鈥,鈥 and 鈥.鈥 They also block 鈥,鈥 鈥,鈥 and 鈥.鈥
  • Clemson University , a meme about a gorilla that had previously generated controversy on campus when a residential employee sought to bar students from referencing the meme on their dormitory doors. Clemson also blocked phrases referring to other controversies on campus, including the name of a professor criticized for calling Republicans 鈥,鈥 the word 鈥溾 (apparently because of a student鈥檚 poll of his classmates, which was reported in the press, about purposefully driving over turtles). Clemson also blocks the words 鈥溾 and 鈥溾濃攁 company with which the university has entered into a $58 million contract.[72] Similarly, the university blocked the words 鈥,鈥 鈥,鈥 and 鈥,鈥 likely relating to students who protested by sitting during the pledge of allegiance during a student government meeting.[73] Clemson also blocked the word 鈥,鈥 referring to who called on the university to end its practice of releasing balloons during football games.[74] Ominously, Clemson also blocked the words 鈥溾 and 鈥.鈥 In response to 果冻传媒app官方鈥檚 records request, Clemson removed the blocked terms and implemented 鈥渕easures so that blocks of protected speech do not occur鈥 in the future.[75]
  • The once-popular awareness campaign 鈥溾[76] cannot be found on Auburn University鈥檚 Facebook page, because comments containing that word are automatically deleted.
  • The University of Mississippi prohibits mentions of its former mascot, 鈥,鈥 a 鈥渃aricature of an antebellum Southern plantation owner鈥 no longer used by the university.[77]
  • The University of Washington blocks a pejorative pun on its team name, the Huskies (鈥溾), as well as references to its football rivals: 鈥,鈥 鈥,鈥 鈥,鈥 and 鈥.鈥 Texas A&M University blocks references to the 鈥溾 horn gesture of their University of Texas at Austin rivals, the Longhorns. Oklahoma State University blocks mentions of its rival football team, including the phrases 鈥溾 (OU鈥檚 fight song), 鈥,鈥 and 鈥.鈥
  • Not to be outdone by its rival, the University of Oklahoma an emoji: ?.[78]
  • The University of Central Florida blocks the words 鈥溾 and 鈥.鈥
  • Oakland Community College in Michigan blocks mentions of the name of a disgruntled former professor.
  • Mississippi State University blocks the term 鈥,鈥 a play on its 鈥淗ail State鈥 motto.[79]
  • A number of institutions, including , , the , the , blocked the names of political candidates, such as 鈥淭rump,鈥 鈥淏ernie,鈥 or 鈥淗illary.鈥 Others, such as , restrict pejorative versions of their names or supporters, such as 鈥淭rumptards鈥 and 鈥淣Obama.鈥
  • West Virginia University blocks mentions of 鈥,鈥 a tradition long disfavored by the university鈥檚 administration and Morgantown residents.[80]

Not just words: Public universities block thousands of Facebook and Twitter users.

FIRE鈥檚 records requests reveal that of the 198 public colleges and universities that responded to 果冻传媒app官方鈥檚 survey, 173 (or 87%) blocked users on Facebook or Twitter. Together, these institutions blocked 13,197 Facebook users and 4,065 Twitter users from interacting with their posts, pages, or tweets. Accordingly, an institution that blocks at least one user blocks an average of 76 Facebook users and 23.5 Twitter users.

Because Facebook and Twitter do not keep records of the reason for blocking a particular account, and because institutions鈥 records reveal only the name of the blocked user (not what they posted), it鈥檚 unclear how many of these users were blocked for viewpoint-discriminatory reasons. It鈥檚 possible many were blocked for violating constitutionally-defensible regulations.

However, government actors cannot restrict access to a public forum based on an objection to the viewpoint of the speaker. Many blocked accounts relate to political, social, or religious movements, including many critical of the institution, raising the possibility that they were blocked for advocating a particular viewpoint.

For example:

  • The University of Kansas blocks 鈥淏oycott Koch Industries,鈥 a Facebook account referencing America鈥檚 second-largest privately-held company,[81] which is based in Kansas and is a major donor to the university.[82] On Twitter, the university blocks accounts related to the Westboro Baptist Church, which had the university鈥檚 graduation.
  • Georgia State University blocked a 鈥溾 Twitter account. The university also about a tweet from the account criticizing the university鈥檚 police the tweet listed the public contact information of the university鈥檚 president and police chief.
  • The University of New Hampshire blocked a Twitter account belonging to 鈥,鈥 a Libertarian Party presidential candidate, and blocked the Twitter account of Breitbart News鈥攁nd other 鈥攁fter the outlet wrote about professors who called for the expulsion of counter-protesters who dressed as Harambe and Richard Nixon.[83] Other accounts appear to have been blocked the university for employing Seth Abramson, whose Twitter criticisms of President Trump have gone viral.
  • The University of Montana blocked a Twitter account,, which was being used to follow 鈥渢he #UMRape crisis that has been plaguing鈥 the university.
  • Mississippi State University blocks 鈥.鈥
  • A number of universities block accounts supporting the 鈥淥ccupy鈥 movement. For example, Ball State University blocks 鈥,鈥 Ivy Tech Community College in Indiana blocks 鈥,鈥 the University of Maryland-College Park blocks 鈥,鈥 and the University of North Carolina blocks 鈥溾 on Facebook.
  • Idaho State University a Facebook page critical of a university employee whose photographs of big game hunting in South Africa went viral.[84]
  • The University of Mississippi, which blocks mentions of its former 鈥淐olonel Reb鈥 mascot on its Facebook page, also a operated by an organization (which also has a registered student chapter at the university) that advocates for the mascot鈥檚 return.[85]
  • The University of Utah blocks animal rights activists, including , former University of Utah student , and 鈥.鈥 Activists, including PETA and Beckham, have criticized research conducted at the university鈥檚 labs.[86]
  • The University of Alaska Anchorage an 鈥淎laskans4Trump鈥 account that had disparaged activist Shaun King in response to a tweet announcing that King would be speaking at the university.[87]

The reason users have been blocked is not always clear from the records produced to 果冻传媒app官方. (The University of North Dakota, for example, blocks a Facebook group entitled 鈥淒og Enthusiasts of North Dakota, no posers, only real dog lovers.鈥) Most institutions block only a handful of users, blocking an average of 76 Facebook users and 23.5 Twitter users. Others wield the tool more liberally. Indiana University blocks the most Facebook users (1488), trailed closely by the University of Washington (1162), with the University of California, Los Angeles in a distant third (691). When it comes to blocked Twitter users, Portland State University blocks the most users (883), more than double that of Central Connecticut State University (307) and Arizona State University (253).

The extent of manual removal of posts and comments is unknown.

FIRE鈥檚 survey did not reveal information about the extent to which institutions utilize their ability to manually hide individual Facebook posts or comments after they鈥檝e been posted. However, if institutions are utilizing tools to prevent content from being posted, it is certain that they are also removing content after it has been posted. Recall, for example, that at Ohio鈥檚 Wright State University, a public records request revealed that administrators hid hundreds of posts concerning an ongoing faculty union strike.[88] Unfortunately, this type of content removal is difficult to detect through broad public records requests.

How private and government actors can mitigate the risk of censoring speech on social media

Government actors and social media companies navigating the application of the First Amendment to the digital realm can both take steps to avoid infringing on fundamental civil liberties.

Public actors and private colleges must avoid censoring speech on social media.

For administrators and staff at public universities and colleges鈥攁s well as at private institutions that promise freedom of expression to students and faculty鈥攖here are a variety of ways to reduce the risk of censorship.

For example:

  • Don鈥檛 filter. Turn off Facebook鈥檚 filters.
  • Don鈥檛 block users. Don鈥檛 block users based on their viewpoint. If they repeatedly violate a policy, document it and give them an opportunity to contest any limits on their access.
  • Craft clear, publicly available policies. Be clear about the purpose of your social media pages. Is your page open for comments only from students or faculty, or is it open to the public? If you place limits on the content of speech, make sure they鈥檙e published, reasonable, narrow, and objective. A vague policy that leaves your staff with a wide range of discretion is a recipe for trouble. Some universities have begun to formulate new policies.[89]
  • Consistently enforce policies once in place. A failure to enforce policies consistently may lead to First Amendment liability even if the policy itself is clear, objective, and reasonable. Selective enforcement will lead to the appearance of bias, and may demonstrate unlawful viewpoint discrimination.[90] Further, at least one court has determined that, because policies were inconsistently enforced, a Facebook page was a designated public forum, not a limited public forum, meaning that it was subject to a stricter First Amendment analysis.[91]
  • Train staff on First Amendment principles. Make sure your staff members tasked with regulating and responding to online speech know what they can and can鈥檛 do, and make sure they ask for help when they鈥檙e not sure.[92]
  • Keep records. If you block a user for posting the same comment repeatedly, or for using unprotected speech, keep a record of it. This will facilitate supervision and correction if a user is blocked for an improper reason.

Social media companies can discourage government actors from abusing tools to censor online speech.

For example:

  • Distinguish between government and non-government accounts. If a user identifies themselves as a government actor, make that information public, and require government officials to designate themselves as such in order to create verified pages or accounts.
  • Require government actors to establish public-facing policies. Restrict government actors鈥 moderation tools unless they have created a public policy concerning user content.
  • Keep filters 鈥渙ff鈥 by default, and only permit their use if the government actor has established a public-facing content policy.
  • Keep filters 鈥渙ff鈥 by default, and only permit their use if the government actor has established a public-facing content policy.
  • Alert users when their comment has been filtered on a government actor鈥檚 page. If a user does not know that their speech has been hidden, they cannot remedy that censorship.
  • Make public the words contained on the profanity filters. Government actors should not be able to make use of a list of forbidden words hidden from the public.
  • Provide tools and reminders for government accounts. If an account is designated as belonging to a government actor, provide warnings alongside filtering or blocking tools that their use may have ramifications under the First Amendment. If a government actor blocks a user, require them to set forth their reason for doing so.

Conclusion

Freedom of expression is often one of the few tools available to people to defend themselves. As more of our public discourse takes place online, and not on the sidewalk or in the pages of newspapers, it is critical that public and private institutions alike vigorously defend expressive rights, and remain vigilant to the impact censorship will have on individuals, groups, and communities.

FIRE stands ready to assist students, student organizations, faculty members, and faculty organizations in defending their rights鈥攐nline or off. If you believe your rights may be jeopardized, contact FIREat /alarm.


Footnotes

[1] Mark Zuckerberg, Standing for Voice and Free Expression, Georgetown University (Oct. 17, 2019) (transcript available at ).
[2] This limitation has been upheld by the Supreme Court. , 569 U.S. 221, 224 (2013).
[3] Facebook, Moderate Your Facebook Page, Dec. 7, 2015, ; see also Facebook, How can I proactively moderate content published by visitors on my Page?, (last visited Apr. 15, 2020).
[4] Id.
[5] Id.
[6] Facebook, How do I ban or unban someone from my Page?, (last visited Apr. 15, 2020).
[7] Twitter, How to control your Twitter experience, https://help.twitter.com/en/safety-and-security/control-your-twitter-experience (last visited Apr. 15, 2020).
[8] This function does not infringe on a user鈥檚 First Amendment rights: One has a right to speak at or about the government, but the government is not obligated to pay attention to that speech. Accordingly, 果冻传媒app官方鈥檚 survey did not ask about muted users or content.
[9] , 137 S. Ct. 1730, 1735 (2017); see also , 928 F.3d 226, 237 (2d Cir. 2019) (鈥淜night鈥) (鈥淸S]ocial media is entitled to the same First Amendment protections as other forms of media.鈥).
[10] To be sure, the First Amendment does not restrict the use of Facebook or Twitter by private actors, such as businesses or private colleges. Nor do Facebook or Twitter themselves violate the First Amendment by providing these tools.
[11] See, e.g., ., 921 F. 3d 440, 447鈥49 (5th Cir. 2019) (assuming a sheriff鈥檚 Facebook page is a public forum and holding that a policy of deleting 鈥渋nappropriate鈥 comments was viewpoint discriminatory); , 912 F.3d 666, 681鈥87 (4th Cir. 2019) (county official鈥檚 Facebook page was a public forum); , No. 18-3485, 2018 U.S. App. LEXIS 36090, at *9鈥14 (6th Cir. 2018) (unpublished opinion reversing sua sponte dismissal of complaint alleging viewpoint discrimination on city鈥檚 official Facebook page); Report and Recommendations, , No. A-19-cv-0198-LY-SH, *11鈥12 (W.D. Tex. Feb. 5, 2020), ECF No. 36 (recommending denial of motion to dismiss First Amendment claim); , No. 4:19-cv-17, at *43鈥44 (W.D. Ark. Sept. 30, 2019), ECF No. 27 (plaintiffs had 鈥渇air chance鈥 of showing state legislator鈥檚 Twitter and Facebook blocks were viewpoint discrimination); , No. 17-cv-2215-W (JLB), U.S. Dist. LEXIS 167247, at *14鈥21 (S.D. Cal. Sept. 26, 2019), ECF No. 42 (denying motion for summary judgment and finding that the interactive portions of school board members鈥 Facebook and Twitter accounts were designated, not limited, public forums); Order of Dismissal, , No. 19-cv-01367 (D. Colo. Sept. 10, 2019), ECF No. 27 (state legislator agrees to unblock Facebook and Twitter critic, pay $25,000); Findings of Fact and Conclusions of Law, , No. 2:18-cv-4129-BCW, 2019 U.S. Dist. LEXIS 138881, at *9鈥15 (W.D. Mo. Aug. 16, 2019), ECF No. 55 (interactive space created by state representative鈥檚 tweets was a designated public forum); , No. 1:19-cv-24, 2019 U.S. Dist. LEXIS 95080, at *13鈥18 (N.D.W. Va. June 6, 2019), ECF No. 18 (First Amendment challenge survives motion to dismiss where constituent blocked from legislator鈥檚 鈥減olitician鈥 Facebook page); , No. 2:18-cv-02138, 2019 U.S. Dist. LEXIS 90509, at *4鈥5 (E. D. Ark. May 30, 2019), ECF No. 17 (interactive portion of police department鈥檚 Facebook page was not government speech); , 354 F. Supp. 3d 940 (W.D. Wis. Jan. 18, 2019) (legislators blocking critic on Twitter); , No. 4:18-cv-01547 (S.D. Tex. Sept. 10, 2018), ECF No. 31 (summary order denying motion to dismiss First Amendment claims premised on keyword-based content filters on public university鈥檚 Facebook page); , No. 17-cv-00296, 2018 U.S. Dist. LEXIS 146894, *36鈥43 (D. Me. Aug. 29, 2018) (governor鈥檚 Facebook page was limited public forum); , No. 15-cv-5871, 2018 U.S. Dist. LEXIS 105815, *25鈥46 (S.D.N.Y. June 25, 2018) (where NYPD precinct blocked Twitter user, whether the forum was a 鈥減ublic, designated, or nonpublic forum鈥 was immaterial, as 鈥渧iewpoint discrimination that results in the intentional, targeted expulsion of individuals . . . is unlawful in any forum,鈥 including nonpublic forums); , No. 3:17-cv-01531, 2018 U.S. Dist. LEXIS 95832 (D. Conn. June 7, 2018) (critic blocked from police Facebook page); , No. 1:16-cv-180, 2017 U.S. Dist. LEXIS 4348, at *10 (E.D. Va. Jan. 10, 2017) (county鈥檚 policy and practice of encouraging comments on its Facebook page created a limited public forum).
[12] , 302 F. Supp. 3d 541, 549 (S.D.N.Y. May 23, 2018) (upheld on appeal by the Second Circuit in Knight).
[13] See, e.g., , No. 18-15712, 2020 U.S. App. LEXIS 5903, at *7 (9th Cir. Feb. 26, 2020) (no First Amendment claim that YouTube 鈥渃ensored鈥 speech because the free speech clause 鈥減rohibits the government鈥攏ot a private party鈥攆rom abridging speech.鈥).
[14] Aaron H. Caplan, , 46 Willamette L. Rev. 647, 647 (2009-2010).
[15] , 333 F.3d 1092, 1099 (9th Cir. 2003) (the 鈥渜uintessential traditional public forums are side-walks, streets, and parks.鈥).
[16] See, e.g., , 586 F.3d 427, 432 (6th Cir. 2009) (school board meetings give 鈥渃itizens a chance to express their views to the board, [but] cannot accommodate the sort of uninhibited, unstructured speech that characterizes a public park.鈥).
[17] , 420 U.S. 546, 555 (1975) (government-leased theater 鈥渦nder [the] control鈥 of public officials); , 518 U.S. 727, 749 (1996).
[18] Government actors have sought to frame 惫颈蝉颈迟辞谤蝉鈥 Facebook or Twitter comments as government speech, arguing that because the government hasn鈥檛 removed those comments, it is effectively publishing the comments as the government鈥檚 own speech. Most people, however, would not view a Facebook comment or tweet as government-endorsed simply because it hasn鈥檛 been removed.
[19] United States v. Alvarez, 567 U.S. 709, 709 (2012).
[20] , 605 F.3d 703, 710 (9th Cir. 2009).
[21] Rosenberger v. Rector & Visitors of Univ. of Va., 515 U.S. 819, 829鈥30 (1995). This is true even if the interactive space of the social media page is deemed a non-public forum, as access even to a non-public forum may only be 鈥渞estricted as long as the restrictions are 鈥榬easonable and [are] not an effort to suppress expression merely because public officials oppose the speaker鈥檚 view.鈥欌 Cornelius v. NAACP Legal Def. & Educ. Fund, Inc., 473 U.S. 788, 800 (1985) (quoting Perry Educ. Ass鈥檔 v. Perry Local Educators鈥 Ass鈥檔, 450 U.S. 37, 46 (1983)).
[22] Good News Club v. Milford Cent. Sch., 533 U.S. 98, 106 (2001) (distinguishing 鈥渢raditional or open鈥 public forums from 鈥渓imited鈥 public forums); , No. 17-cv-2215-W (JLB), U.S. Dist. LEXIS 167247, at *14鈥21 (S.D. Cal. Sept. 26, 2019), ECF No. 42 (interactive portions of school board members鈥 Facebook and Twitter accounts were designated, not limited, public forums); , 354 F. Supp. 3d 940, 953鈥55 (W.D. Wis. Jan. 18, 2019) (interactive portions of legislators鈥 Twitter accounts were designated public forums).
[23] Perry Educ. Ass鈥檔, 450 U.S. at 45.
[24] Id.
[25] , U.S. Dist. LEXIS 167247 at *20鈥21.
[26] , 480 F. 3d 891, 908 n.8 (9th Cir. 2006) (A 鈥渓imited public forum is a sub-category of the designated public forum, where the government opens a nonpublic forum but reserves access to it for only certain groups or categories of speech.鈥).
[27] Good News Club, 533 U.S. at 106鈥07 (quoting, in part, Rosenberger v. Rector & Visitors of Univ. of Va., 515 U.S. 819, 829 (1995)).
[28] , 699 F.3d 1053, 1062 (9th Cir. 2012).
[29] See, e.g., , 482 F.Supp.2d 1003, 1009卢卢鈥11 (S.D. Ohio 2007) (government agency鈥檚 bulletin board had 鈥渟ome aspects of a limited public forum鈥 because it was 鈥渙pen to all employees鈥 to 鈥減ost matters addressing a broad range of topics or meetings,鈥 rendering removal of religious materials unconstitutional).
[30] Rosenberger v. Rector & Visitors of Univ. of Va., 515 U.S. 819, 830 (1995).
[31] Id.
[32] Perry Educ. Ass鈥檔 v. Perry Local Educators鈥 Ass鈥檔, 450 U.S. 37, 46卢鈥49 (1983).
[33] , 605 F.3d 703, 710 (9th Cir. 2009).
[34] Compare , 221 F.3d 834, 844 (6th Cir. 2000) (city鈥檚 website listing links to other sites was a non-public forum, not a public forum, because it was not structured to allow 鈥渄ialogue between users,鈥 but primarily 鈥渢o convey information to the reader鈥) with , 531 F.3d 275, 284 (4th Cir. 2008) (school district鈥檚 website was not a public forum, but 鈥渢he issue would, of course, be different鈥 if the website were a 鈥渢ype of 鈥榗hat room鈥 or 鈥榖ulletin board鈥 in which private viewers could express opinions or post information鈥).
[35] , 302 F. Supp. 3d 541, 572鈥73 (S.D.N.Y. May 23, 2018).
[36] , 912 F.3d 666, 687鈥88 (4th Cir. 2019) (official who blocked constituent because of his critical viewpoint was viewpoint discrimination, which is 鈥減rohibited in all forums.鈥).
[37] , 921 F. 3d 440, 447 (5th Cir. 2019).
[38] at 238鈥39.
[39] , 247 F.Supp.3d 767, 777 (E.D. Va. 2017).
[40] See, e.g., , 990 F.2d 586, 591 (11th Cir. 1993) (a restriction vesting unfettered discretion in a government actor 鈥渙pens the way to arbitrary suppression of particular points of view.鈥).
[41] Greg Harold Greubel, Public records reveal Wright State used unconstitutional Facebook page policy to censor pro-union speech during historic faculty strike, Found. for Indiv. Rights in Educ., Jan. 24, 2020, /public-records-reveal-wright-state-used-unconstitutional-facebook-page-policy-to-censor-pro-union-speech-during-historic-faculty-strike; see also, Letter from Greg Harold Greubel, Staff Attorney, Foundation for Individual Rights in Education to Larry Y. Chan, General Counsel, Wright State University, Jan. 23, 2020, available at /fire-letter-to-wright-state-university-january-23-2020.
[42] Healy v. James, 408 U.S. 169, 180 (1972) (there is 鈥渘o room for the view that . . . First Amendment protections should apply with less force on college campuses than in the community at large鈥).
[43] As discussed in the methodology section of this report and in the appendices, some states or institutions could not be surveyed, failed to respond at all, sought exorbitant fees, or cited exceptions to open records laws, ranging from arguments that they cannot be required to 鈥渃reate鈥 records to claims that revealing Facebook blacklists creates a risk to public safety.
[44] See, Miami Herald Pub. Co., Div. of Knight Newspapers Inc. v. Tornillo, 418 U.S. 241, 256鈥57 (1974) (a 鈥淸g]overnment-enforced right of access鈥 in the form of a statute compelling newspapers to publish rebuttals 鈥渙perates as a command in the same sense as鈥 a rule prohibiting speech, violating the First Amendment). This right is fortified by statute shielding providers of an 鈥渋nteractive computer service鈥 from civil liability for removing content they deem 鈥渙bjectionable.鈥 .
[45] Cohen v. California, 403 U.S. 15, 16 (1971).
[46] Id. at 26.
[47] Texas v. Johnson, 491 U.S. 397, 414 (1989).
[48] Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A. Smith, 2019, The Risk of Racial Bias in Hate Speech Detection. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics at 1668鈥78, available at .
[49] Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2018. Measuring and Mitigating Unintended Bias in Text Classification. In Proceedings of AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society at 67, available at .
[50] Thomas Davidson, Debasmita Bhattacharya, and Ingmar Weber, 2019, Racial Bias in Hate Speech and Abusive Language Detection Datasets, available at .
[51] Mike Masnick, Masnick鈥檚 Impossibility Theorem: Content Moderation At Scale Is Impossible To Do Well, Techdirt, Nov. 20, 2019, .
[52] Shirin Ghaffary, The algorithms that detect hate speech online are biased against black people, Vox, Aug. 15, 2019, .
[53] See, e.g., Matal v. Tam, 137 S. Ct. 1744, 1763 (2017) (striking down regulation barring trademark registration for 鈥渁ny mark that is offensive to a substantial percentage of the members of any group鈥 because it amounted to viewpoint discrimination, and observing that 鈥淸g]iving offense is a viewpoint鈥).
[54] Grayned v. City of Rockford, 408 U.S. 104, 108 (1972).
[55] Neb. Press Ass鈥檔 v. Stuart, 427 U.S. 539, 559 (1976).
[56] Near v. Minnesota, 283 U.S. 697, 713 (1931).
[57] Shuttlesworth v. Birmingham, 394 U.S. 147, 151 (1969).
[58] See, e.g., , No. 3:18-cv-1017 (PAD), 2018 U.S. Dist. LEXIS 208014, at *21鈥24 (D.P.R. Dec. 10, 2018) (surveying the procedural and substantive strands of the doctrine of prior restraint); see also, , 493 F.3d 421, 431 (4th Cir. 2007) (鈥淎 prior restraint on speech that imposes no time limitations on the decision-making process plainly fails鈥 First Amendment scrutiny when the prior restraint is based on content.).
[59] In the Matter of a Citizen鈥檚 Complaint Against Pacifica Foundation Station WBAI (FM), New York, New York, 56 F.C.C.2d 94, 109, Declaratory Order (Feb. 21, 1975) (Transcript of George Carlin鈥檚 鈥淔ilthy Words鈥 monologue, prepared by the Federal Communications Commission), archived at .
[60] See, e.g., Univ. of N.C. at Chapel Hill, Recognizing Carolina鈥檚 black pioneers, Facebook (Feb. 23, 2018), .
[61] After a letter from 果冻传媒app官方, UNC removed some of the terms on its customized list, but it is not clear what terms remain. Letter from Kara E. Simmons, Associate Vice Chancellor and Senior Univ. Counsel, Univ. N.C., to Adam Steinbaugh, Director, Individual Rights Defense Program, Found. for Indiv. Rights in Educ. (Sept. 13, 2019) (on file with author).
[62] Univ. of N.C. at Chapel Hill, Chancellor Folt announces resignation. . ., Facebook (Jan. 15, 2019), .
[63] Univ. of N.C. at Chapel Hill, Carolina community gathers in remembrance of 9/11, Facebook (Sept. 11, 2018), .
[64] See, e.g., Oklahoma State Univ., It鈥檚 Election Day, Cowboys! Get out and VOTE!, Facebook (Nov. 8, 2016), .
[65] Good News Club v. Milford Cent. Sch., 533 U.S. 98, 106鈥07 (2001) (quoting, in part, Rosenberger v. Rector & Visitors of Univ. of Va., 515 U.S. 819, 829 (1995)).
[66] Jay Blanton, Dining Partnership Will Transform Vital Service to UK Campus, Univ. of Ky., June 13, 2014, .
[67] Adam Schwartz, EFF Lawsuit Ends Censorship Against PETA on Public University鈥檚 Facebook Page, Electronic Frontier Found., Feb. 4, 2020, ; see also Adam Steinbaugh, Texas A&M: Our secret list of naughty words you can鈥檛 say on our Facebook page doesn鈥檛 offend the First Amendment, Found. for Indiv. Rights in Educ., Aug. 23, 2018, /texas-am-our-secret-list-of-naughty-words-you-cant-say-on-our-facebook-page-doesnt-offend-the-first-amendment.
[68] Press Release, Secret Student Video Exposes Dissection of Pregnant Cat at Santa Monica College, People for Ethical Treatment of Animals, Sept. 7, 2016, .
[69] Neel Ahuja, Distorting the study of 9/11 at UNC, News & Observer, Sept. 19, 2015, .
[70] Jane Stancill, UNC course on 9/11 criticized in conservative publications, Charlotte Observer, Sept. 1, 2015, .
[71] Aviva Shen, University Of Arizona Student Tells Women: 鈥榊ou Deserve Rape鈥, ThinkProgress, Apr. 25, 2013, .
[72] Manie Robinson, Clemson extends Nike partnership with 10-year, $58 million deal, Greenville News, Aug. 3, 2018, .
[73] Georgie Silvarole, Clemson University student senators sit for Pledge of Allegiance, WCNC, Sept. 28, 2017, .
[74] Matt Connolly, Clemson ending this football gameday tradition for 2018, The State, July 28, 2018, .
[75] E-mail from Jermaine D. Johnson, Assistant General Counsel, Clemson Univ., to Adam Steinbaugh, Director, Individual Rights Defense Program, Found. for Indiv. Rights in Educ. (Dec. 21, 2018, 10:12 PM), available at .
[76] Invisible Children, Kony 2012, (last visited Apr. 15, 2020).
[77] Robbie Brown, Ole Miss Shelves Mascot Fraught With Baggage, N.Y. Times, Sept. 19, 2019, .
[78] Extending the middle finger is expression protected by the First Amendment. , 704 F.3d 105, 111 (2d Cir. 2013).
[79] Jason Kirk, Some moron spray-painted 鈥楩ail State,鈥 鈥楪o Rebs,鈥 and more on Mississippi State鈥檚 campus, SBNation, Dec. 17, 2016, .
[80] Jesse Wright, With Hopes to Curb Couch Burning, Morgantown Passes Outdoor Furniture Ban, W.VA. Pub. Broad., Apr. 7, 2015, .
[81] America鈥檚 Largest Private Companies, Forbes, (last visited Apr. 15, 2020).
[82] Press Release, Koch Industries Gift of $1M Establishes Business, Engineering Scholarship Funds, Univ. of Kan. (Sept. 8, 2016), . FIREalso receives funding from the Charles Koch Foundation. FIREdoes not accept money from donors who seek to alter our mission or interfere with our case work.
[83] Katherine Rodriguez, University of New Hampshire Professors Call for Trump Supporters to Be Expelled, Breitbart, Nov. 21, 2016, .
[84] Michael H. O鈥橠onnell, ISU accountant on safari becomes social media target, Idaho St. J., Aug. 5, 2015, .
[85] Colonel Reb Found., About Us, (last visited Apr. 15, 2020); see also Univ. of Miss., Colonel Reb Found., (recognized student organization webpage) (last visited Apr. 15, 2020).
[86] Courtney Tanner, University of Utah can鈥檛 charge PETA a $5K 鈥榩repayment鈥 for records on lab animal deaths, committee rules, Salt Lake Trib.., Nov. 14, 2018, .
[87] @Alaskans4Trump, Twitter (Feb. 7, 2017, 1:30 AM), .
[88] Greg Harold Greubel, Public records reveal Wright State used unconstitutional Facebook page policy to censor pro-union speech during historic faculty strike, Found. for Indiv. Rights in Educ., Jan. 24, 2020, /public-records-reveal-wright-state-used-unconstitutional-facebook-page-policy-to-censor-pro-union-speech-during-historic-faculty-strike.
[89] See, e.g., and the .
[90] Thomas v. Chi. Park Dist., 534 U.S. 316, 325 (2002) (granting 鈥渨aivers to favored speakers鈥 or 鈥渄enying them to disfavored speakers鈥 would 鈥渙f course be unconstitutional鈥); see, e.g., , 360 F.Supp.3d 885, 907 (S.D. Iowa 2019) (disparate enforcement of nondiscrimination policy violated the First Amendment).
[91] , No. 17-cv-2215-W (JLB), U.S. Dist. LEXIS 167247, at *20 (S.D. Cal. Sept. 26, 2019), ECF No. 42.
[92] To its credit, Wayne State University started training its social media staff after receiving a records request from 果冻传媒app官方.


Appendices

Appendix A

Surveyed institutions and response data

You can review the survey results, including links to each institution's records, here: .

Institutions in blue provided adequate responses. Copies of the records are hyperlinked.

Institutions marked in red failed to produce sufficient documents for the reasons specified in the accompanying footnote or hyperlinked document.

Appendix B

Exceptions to the survey methodology

The survey sought to cover two sets of institutions within each state, plus the District of Columbia: the three four-year institutions with the highest enrollment and the two two-year institutions with the highest enrollment. This would amount to 255 institutions, but not every state has qualifying institutions, some states鈥 public records laws do not extend to those institutions, and some institutions cannot be surveyed due to a legal conflict. What follows is a list of states or institutions falling under an exception or conflict:

  • Arkansas: Public records law limited to citizens of Arkansas. (-5 institutions).
  • California: East Los Angeles College could not be contacted due to litigation conflict. Replaced by American River College.
  • Delaware:
    • Only two four-year institutions, and only one two-year institution qualified. (-2 institutions).
    • The qualifying four-year institutions (Delaware State University and the University of Delaware) are . (-2 institutions).
  • Iowa: Iowa State University could not be contacted due to litigation conflict. Cannot be replaced by another four-year institution because there are only three public four-year institutions in Iowa. (-1 institution).
  • Louisiana: Louisiana State University could not be contacted due to litigation conflict. Replaced by Louisiana Tech.
  • New Jersey: Rutgers University could not be contacted due to possibility of litigation conflict. Replaced with Rowan University.
  • Pennsylvania: The Right to Know law largely exempts four-year institutions, but does not exempt two-year institutions. (-3 institutions).
  • Rhode Island:
    • The Naval War College was not surveyed because its relationship with the armed services may change the First Amendment calculus. It was not replaced because there are only three, four-year institutions in the state. (-1 institution).
    • Rhode Island only has one two-year institution. (-1 institution).
  • South Carolina: The University of South Carolina could not be surveyed due to litigation conflict. Replaced by Coastal Carolina University.
  • Tennessee: FOIA limited to citizens of Tennessee. (-5 institutions).
  • Virginia: FOIA limited to citizens of Virginia. (-5 institutions).
  • Washington, D.C.:
    • There鈥檚 only one public, four-year institution in Washington, D.C., that is not affiliated with the military. (-2 institutions).
    • There are no public two-year institutions. (-2 institutions).
  • Wyoming: There is only one public, four-year institution. (-2 institutions).

Appendix C

Example public records request

The following is a lightly edited version of the request sent to Michigan State University:

This is a request for records pursuant to the Michigan Freedom of Information Act (Mich. Comp. Laws Ann. 搂搂 15.231 et seq.). If you are not the records custodian for MSU, please identify the correct person to contact.

This request seeks records relating to restrictions and settings concerning the official Facebook and Twitter accounts for MSU, and should be directed to the person responsible for operating those accounts.

RECORDS REQUESTED

I request the following records:

  1. A copy of the settings for the Facebook page maintained by Michigan State University (available at https://www.facebook.com/spartans.msu). This list is accessible by (A) logging into the Facebook page as an administrator, and then (B) clicking 鈥淪ettings鈥 at the top of the official page. The URL should look like: https://www.facebook.com/spartans.msu/settings/?tab=settings.
  2. A copy of the list of people or pages banned from the Facebook page referenced above. This list is accessible by: (A) logging into the Facebook page as an administrator, (B) clicking 鈥淪ettings鈥 at the top of the official page, (C) clicking 鈥淧eople and Other Pages鈥 in the left column, and (D) selecting "Banned People and Pages" from the drop-down menu. The final URL should look like: https://www.facebook.com/spartans.msu/settings/?tab=people_and_other_pages.
  3. A list of the "blocked accounts" by the Twitter account maintained by Michigan State University (available at https://twitter.com/michiganstateu). This list is accessible by navigating to this URL while logged into the account: https://twitter.com/settings/blocked.

Fee waiver request: This request concerns a matter of public interest. The social media restrictions imposed by public institutions and officials 鈥 including the president, governors, and public universities 鈥 have been challenged on First Amendment grounds. This survey seeks to explore the extent to which public colleges and universities have similar restrictions.

The public interest would be well-served by granting a fee waiver. The request is not being sought for a commercial purpose, but is instead sought by a nonprofit organization to provide the public with information concerning the conduct of government actors as that conduct pertains to civil liberties in higher education.

If a fee waiver is not granted, please apprise me if the estimated costs will exceed $10.

Request for expedited processing: Completion of this survey depends on the institution with the slowest response time. We request that Michigan State University produce responsive records on an expedited basis. As you may be aware, a public body has five business days to respond. (Mich. Comp. Laws. Ann 搂 15.235(2)).

Appeal information: If you deny any portion, or all, of this request, please provide me with a written explanation of the reason(s) for your denial, including a citation to each specific statutory exemption you believe justifies the refusal to release the information and notify me of the appeal procedures available to me under the law. If you conclude that portions of the records that I request are exempt from disclosure, please release the remainder of such records for inspection and copying, redacting only the portion or portions that you claim are exempt.

Appendix D

Customized blocked words

The surveyed institutions blocked the following unique words using Facebook鈥檚 customizable blacklist. Duplicates were removed.

100
9/11
9-11
:))
@$$
#cocksoutforharambe
#dicksoutforharambe
<--- 鈾 us 鈾
?
$
$hit
2's!!!
2004toyota 4runner
5g
5t
a$$
a55
a55hole
abcreports
abuse
abusers
accident
aeolus
africant
aggy
agsaf
ahole
ahuja
alert
amazing
amcik
ame law
americanadvantagelawgroup@lawyer.com
anal
analprobe
andskota
anilingus
annoying
anthem
anus
apartment
apologize
apple
appleguru
applelovers
applemusiclist
apps
aramark
areola
areole
arian
arrrrr
arschloch
arse
artificially generated stampede awareness foundation
aryan
ash0le
asholes
asl
asla
ass
ass hole
ass monkey
assault
assbang
assbanged
assbangs
asses
assface
assfuck
assfucker
assh0le
assh0lez
asshat
assho1e
asshoel
asshole
assholery
assholes
assholz
asskisser
asslick
asslicker
asslickers
asslicks
assmaster
assmunch
assrammer
asswipe
asswipes
attention
auto
awaay
awayy
ayir
azazel
azz
azzhole
b-s
b!+ch
b!tch
b*tch
b00bs
b17ch
b1tch
b1tches
babe
babes
baboon
bad
balloon
balloonsblow
balls
ballsack
baloons
bammer
bammers
ban
bang
banger
banned
barf
bart
bassterds
bastard
bastards
bastardz
basterds
basterdz
bawdy
bdsm
beaner
bearded clam
beardedclam
beastial
beastiality
beastility
beat-off
beatch
beater
beating beat sucking
beating-off
beatoff
beaver
beavers
beavs
beecher
beer
beeyotch
before
believee
belly whacker
beotch
bernie
bert
bestial
bestiality
bfd
bi+ch
bi7ch
biatch
biche
big beoyotch
big tits
bigleaguepolitics.com
bigtits
bimbo
bing
bings
birds
birt
bitcard
bitch
bitchass
bitched
bitcher
bitchers
bitches
bitchin
bitching
bitchs
bitchy
blizzard
blocked
blondehot videos
bloody
blow
blow job
blowjob
blowjobs
blue waffle
bod
bodily
boffing
boink
boiolas
bollock
bollocks
bollok
bone
boned
bonehead
boner
boners
bong
boob
boobies
boobs
booby
booger
bookie
boomer
boomer sooner
bootee
bootie
booty
booze
boozer
boozy
bort
bosom
bosomy
boss
bowel
bowels
boycott
bra
brassiere
breast
breasts
britney sprays video
broken
brokenn
brokeon
brokken
brother dean
brown eye
browneye
browntown
bruins
bs
buceta
bucket bull bullshit
bug
bugger
bukkake
bull shit
bull-shit
bullsh*t
bullshit
bullshits
bullshitted
bullturds
bullying
bum
bung
bung hole
burt
business
busty
butch
butt
butt breath
butt fuck
butt-pirate
buttface
buttfuck
buttfucker
butthead
butthole
butthurt
button
buttpicker
buttplug
buttwipe
buy online
byob
bytch
c-0-c-k
c-o-c-k
c-u-n-t
c.0.c.k
c.o.c.k.
c.u.n.t
c0ck
c0cks
c0k
ca-ca
cabron
caca
cahone
call
camel jockey
camelfucker
cameljockey
cameltoe
cancel
canceled
cancelled
cannooot
canoe
carpet muncher
carpetmuncher
cashcars
cawk
cawks
cazzo
centsy
cervix
change org
cheap
check this out
chegg
chicken
chickens
chilango
chinc
chincs
chink
chode
chodes
chola
cholo
chraa
chuj
cialis
cipa
circle jerk
citylife
cl1t
class
cleared
click on advertisement
climax
clinton
clit
clitoris
clitorus
clits
clittie
clitty
close
closed
cnts
cntz
cobia
cocain
cocaine
cock
cock sucker
cock-head
cock-sucker
cock-sucking
cockblock
cockhead
cockholster
cockknocker
cocks
cocksauce
cocksmoker
cocksoutforharambe
cocksuck
cocksucked
cocksucker
cocksucking
cocksucks
coital
col
col.
col. reb
colonel mascot
commie
condom
coochi
coon
coons
cooter
corksucker
cost-free
couch burning
coug
cougars
cougs
cowfuck
cowfucker
cowfuckers
crabs
crack
cracker
crackwhore
crap
crappy
crash
crs
cruel
csc
cuck
cuckservative
cum
cum stain
cumdumpster
cummer
cummin
cumming
cums
cumshot
cumshots
cumslut
cumstain
cunilingus
cunillingus
cunnilingus
cunny
cunt
cuntface
cunthunter
cuntlick
cuntlicker
cuntlicking
cuntpunch
cuntpuncher
cunts
cuntsucker
cunty
cuntz
cya
cyberfuc
cyberfuck
cyberfucked
cyberfucker
cyberfuckers
cyberfucking
d's
d0ng
d0uch3
d0uche
d1ck
d1ld0
d1ldo
d4mn
dago
dagos
damage
dammit
damn
damned
damnit
dangerous
dawgie-style
daygo
death
deez
defriended
dego
delay
derka
deserve youtube
deviant
diaper double d
dic
dick
dick-ish
dickbag
dickdipper
dickface
dickflipper
dickhead
dickheads
dickish
dickless
dickripper
dicks
dicksipper
dickwad
dickweed
dickwhipper
dickzipper
diddle
die
dies
dike
dild0
dild0s
dildo
dildos
diligaf
dilld0
dilld0s
dillweed
dilweed
dimwit
dingle
dingleberry
dining services
dink
dinks
dipship
dipshit
dirsa
disckwad
discount
disgrace
diversity room
dkendrick
doggie dirty sanchez
doggie-style
doggy style
doggy-style
doingg
doinng
dominatricks
dominatrics
dominatrix
dong
doofus
doosh
dope
dopey
dotard
douch3
douche
douche bag
douchebag
douchebags
douchecanoe
douches
douchey
drive
drumpf
drunk
ducks
dumass
dumbass
dumbasses
dummy
dupa
durka
dyke
dykes
dyko
dziwka
eff
effin
effs
ejackulate
ejactulated
ejaculate
ejaculates
ejaculating
ejaculatings
ejaculation
ejakulate
ekrem
ekto
embarassing
embarrassing
emergency
enculer
enema
enlargement
enter to win
erect
erection
erotic
error
esad
essohbee
extacy
extasy
f u c k
f-u-c-k
f.u.c.k
f**cking
f**k
f**king
f*cking< f*ck
facebook.com
fack
faen
fag
fag1t
faget
fagg
fagg1t
fagged
fagget
fagging
faggit
faggot
faggots
faggott
faggotty
faggoty
faggs
faggy
fagit
fagot
fagots
fags
fagz
faig
faigs
faigt
fail state
failstate
fairy
fanculo
fanny
fannybandit
fart
farted
farting
fartknocker
farts
farty
fat
fat fatass
fatso
fcuk
feces
feck
feg
felatio
felch
felcher
felching
fellate
fellatio
feltch
feltcher
feltching
feminazi
ficken
filthy
fingerfuck
fingerfucked
fingerfucker
fingerfuckers
fingerfucking
fingerfucks
fire
fishbucket
fisted
fistfuck
fistfucked
fistfucker
fistfuckers
fistfucking
fistfuckings
fistfucks
fisting
fisty
fitt
flange
flikker
flipping bird
floozy
foad
foag
fondle
foobar
food
forefreedelivered
foreskin
fotze
fournier
free
freee
freeed
freeee
freex
friedcatz
frigg
frigga
fu
fu(
fu*k
fubar
fuck
fuck-tard
fuckass
fucked
fucker
fuckers
fuckface
fuckin
fucking
fuckings
fuckme
fucknugget
fucknut
fucko
fuckoff
fuckos
fucks
fucktard
fuckup
fuckwad
fuckwit
fudge packer
fudge-packer
fudge-packing
fudgepacker
fudgepacking
fuk
fukah
fuken
fuker
fukin
fukk
fukkah
fukken
fukker
fukkin
fuks
furburger
fuskies
futkretzn
fux0r
fvck
fxck
g-spot
g00k
gae
gag reflex
gai
gamesplaystoplay.co.cc
gamesplaytoplay
gangbang
gangbanged
gangbanger
gangbangers
gangbangs
ganja
gay
gay sex
gayboy
gayest
gaygirl
gays
gaysex
gayz
gazongers
gey
gfy
ghay
ghey
giftcard
gigolo
gina
gink
giving
givingg
givng
givving
glans
glitch
glitching
glory god goddamn
goatse
God damn
God Goddamn
god-damned
godamn
godamnit
goddam
goddammit
goddamn
goldenshower
gonad
gonads
goo.gl
gook
gooks
gooo
gop
gorilla
gorillalivesmatter
gorunner
gov
govgrantmoney
goyfa
grades
greaser
greee
grinder
gringo
grooniu
grooniu?zcgshpndpzruqjcyurch
groonsiu
gspot
gtfo
guido
guiena
guinne
gypsies
gypsy
h.c.c.
h00r
h0ar
h0m0
h0mo
h0r
h0re
h4x0r
haaha
haha
hahahahha
hahatodayfbs.net.tc
hair
hand fudge packer
hand-job
handjob
harambe
harambe!
harass
hard on
hardcoresex
hardee
harmbe
haughwout
haunt
hcc
he11
hebe
heeb
heil
heimey
hell
hells
helme
helvete
hemp
heree
heroin
herp
herpes
herpy
hershey highway
hey
hey there university
hick
hicks
hiddenpicz
hillary
himey
hitler
hiv
hoar
hobag
hoer
holocaust
hom0
homey
homo
homoey
homophobic
homos
honkey
honkie
honky
hooch
hood rat
hoodrat
hook
hookah
hookem
hooker
hoor
hoore
hootch
hooter
hooters
hore
horniest
horny
horrendous
horseshit
hotsex
housatonic
houston community college
http
http://apps.facebook.com
https
https://soundcloud.com/allnentertainmentt
huevon
hui
hump
humped
humping
hurry
hussy
hymen
i padds
i-paaaad
i-paaaadd
i-paaadd
ice
icrazy
idiocy
idiot
idiots
ifree
igiveaway
iipaaaad
iipaaaadd
iipaaaaddd
iipaaaadddtwos
iipaaaaddtwos
iipaaad
iipaaadd
iipaaadd2s
iipaaaddtwo's
iipaad
iipad
iipad 2's
iipad2
iipad2s
iipad2ss
iipads
iipads2
iippaaaaad2
ike likeeeeee
illuminati
im
inbred
incest
injun
insurance
intercourse
ipaaaad
ipaaad
ipaaad twoooosss
ipaaadd
ipaaadd2
ipaaaddd
ipaad
ipaad2
ipaad2's
ipaads
ipad
ipad2
ipad2entry
ipad2s
ipadd
ipads
ipadss
iphon5
iphone
iphone 5
iphone5
iphones
ippaaaadtwoo
ippaad
ippad
ippads
iq
iq score
irresponsible
isis
j3rk0ff
jack
jack off
jack-off
jackass
jackhole
jacking jackoff
jackoff
jambo
jap
japs
jerk
jerk jerkoff
jerk-off
jerk0ff
jerked
jerkoff
jersey
jersey store
jetblue
jewpidily
jigaboo
jihad
jisim
jism
jiss
jiz
jizm
jizz
jizzed
junkie
junky
k e r
kaffer
kaffir
kanker
kawk
kido
kike
kikes
kill
killary
kills
kinky
kkk
klan
klootzak
kma
kmia
knijnenburg
knob
knob end
knobend
knobs
knobz
knulle
Kock
kondum
kondums
kony
kooch
kooches
kootch
kracker
kraut
kuk
kuksuger
kum
kummer
kumming
kums
kunilingus
kunt
kunts
kuntz
kurac
kurwa
kusi
kyke
kyrpa
l3i+ch
l3itch
laate
lab
labia
lame
lampshade
late
latee
lech
leper
leppo
lesbian
lesbians
lesbo
lesbos
lez
lezbian
lezbians
lezbo
lezbos
lezzian
lezzie
lezzies
lezzy
libtard
libtards
life
like
like pls. :)
linthead
lipshits
lipshitz
listofapples
lmao
lmbo
lmfao
loin
loins
lube
lucky
lusty
m-fucking
Mafucka
mamhoon
mams
manigger
marijuana
masochits
masokist
massa
massterbait
masstrbait
masstrbate
masterbaiter
masterbat
masterbat3
masterbate
masterbates
masterbating
masterbation
masturbat
masturbate
masturbating
masturbation
maxi
md
mears
menses
menstruate
menstruation
merd
merde
messing
messng
messsin
messsing
meth
mexcrement
mexican't
mexicant
mibun
mick
middlesex
migger
minee
mofo
molest
money
monkleigh
moolie
moron
morons
moshky
moss
motehrfucking
motha mother-fucker
mothafuck
mothafucka
mothafuckas
mothafuckaz
mothafucked
mothafucker
mothafuckers
mothafuckin
mothafucking
mothafuckings
mothafucks
mother fucker
motherfuck
motherfucka
motherfucked
motherfucker
motherfuckers
motherfuckin
motherfucking
motherfuckings
motherfucks
motos
motss
mouliewop
mountainqueer
mountainqueers
ms
msu dining
mtherfucker
mthrfucker
mthrfucking
muff
muffdiver
muie
mulkku
murder
muschi
mutha n1gr
muthafuckaz
muthafucker
mutherfucker
mutherfucking
muthrfucking
my erotic videoss
myn
myne
n'er
nad
nads
nagur
naked
napalm
nappy
nastt
naugatuck
nazi
nazis
nazism
neel
negro
negroes
nepesaurio
ner
newinbox
news
newsfeedz
newzfeed
nfw
nice share
nig
nigga
niggah
niggas
niggaz
nigger
niggers
niggle
niggs
night
niglet
nigs
nigur
niiger
niigr
nike
nimrod
ninny
nipple
nobama
nooky
notmanyleft
nowayjoesez
nutsack
nutz
nvcc
nympho
odumbo
offer
omg
oout
open
opiate
opium
orafis
oral
orally
orangeman
organ
orgasim
orgasims
orgasm
orgasmic
orgasms
orgasum
orgies
orgy
oriface
orifice
orifiss
orospu
ou
ovary
ovum
ovums
p.u.s.s.y.
p0rn
packi
packie
packing
packy
pad
paddy
page
pagee
paki
pakie
paky
pantie
panties
panty
park
parking
partyvid
paska
pastie
pasty
pcp
pecker
pedo
pedobear
pedophile
pedophilia
pedophiliac
pedos
pee
peeenus
peeenuss
peenus
peepee
peinus
pen1s
penas
penetrate
penetration
penial
penile
penis
penis-breath
penises
penus
penuus
perse
perversion
peta
petition
peyote
phalli
phallic
phone phonesex
phuc
phuck
phuk
phuked
phuker
phuking
phukked
phukker
phukking
phuks
phuq
picka
picture
pierdol
piicture
pillow biter
pillow-biter
pillowbiter
pillu
pimmel
pimp
pimpis
pinko
piss
piss-off
pissed
pissers
pisses
pissin
pissing
pissoff
pissrr
pita
pizda
pjaavpietrab
pledge
plow
plssss
plug
pmp
pms
polac
polack
polak
pollock
poon
poonani
poontang
poontsee
poop
poor
porn
porno
pornography
pos
pot
potty
ppictureeee
ppiicctureeee
pr0n
pr1c
pr1ck
pr1k
preteen
prick
pricks
prig
profile
prostitute
prude
pube
pubic
pubis
pula
pule
punkass
punky
puss
pusse
pussee
pussies
pussy
pussypounder
pussys
puta
puto
puuke
puuker
qahbeh
queaf
queef
queefs
queer
queero
queers
queerz
quicky
quim
qweers
qweerz
qweir
r-tard
racy
rag head
raghead
ragheads
rape
rapeculture
raped
raper
rapist
rapists
raunch
rautenberg
really
rebel
recieved
recktum
rectal
rectum
rectus
redneck
rednecks
reefer
reetard
reich
republican
resign
retard
retarded
retards
revue
rim-job
rimjob
rimming
ritard
roads
rofll
roflmao
rotflmaotid
rotflolapmp
rtard
rtfm
rtwfq
rum
rump
rumprammer
ruski
s hit
s-h-1-t
s-h-i-t
s-o-b
s.h.i.t.
s.o.b
s.o.b.
s0b
sadism
sadist
sale
sales
sandnigger
scag
scank
scantily
scared
scat
schaffer
scheiss
schizo
schlampe
schlong
schmuck
screw
screwed
screwin
screwing
scrog
scrot
scrote
scrotum
scrud
scrwing
scum
seaman
seamen
seduce
seks
self
sell
selling
semen
senators
sendng
sex
sexual
sexy
sh!+
sh!t
sh*t
sh1t
sh1ter
sh1ts
sh1tter
sh1tz
shamedame
sharmuta
sharmute
sheister
shemale
shi+
shipal
shit
shit face
shite
shiteater
shited
shitface
shitfucker
shitfull
shithead
shithold
shithole
shithouse
shiting
shitings
shits
shitshow
shitt
shitted
shitter
shitters
shittin
shitting
shittings
shitty
shity
shitz
shiz
shot
shyt
shyte
shytty
shyty
silent sam
sissy
site
sitee
skag
skanck
skank
skankee
skankey
skanks
skanky
skeet
skittle
skittles
skrib
skype
skypes
slags
slave
sleaze
sleazy
slip
slut
slutdumper
slutkiss
sluts
slutty
smart
smegma
smut
smuts
smutty
snatch
snigger
sniper
snow
snowflake
snowflakes
snuff
sodom
sol
son-of-a-bitch
sooners
sorry
souse
soused
sperm
spic
spick
spik
spiks
splooge
spooge
sprays
spraysvideo!!!!..
spunk
stalker
steamy
stfu
stiffy
stoned
street
strip
stroke
stupid
suck
sucked
sucking
sucks
sucky
sumofabiatch
sumshots
sutz
t1t
taint
tampon
tard
tards
tawdry
teabagging
teat
teets
teez
terd
terrible
terroists
terrorism
terrorist
teste
testee
testes
testical
testicle
testis
the colonel
theyre
thisi
thiss
thisss
thrust
thug
tinkle
tit
titfuck
titi
tits
titslut
titt
tittiefucker
titties
titty
tittyfuck
tittyfucker
toe
toke
tomorrow
toots
top10 unviersities
tosser
towel-head
towelhead
towhead
tragedy
tramp
trannsy
tranny
transsexual
trashy
trevillian
truck
truebloodx
trump
trumptards
tubgirl
turd
turtle
turtles
tush
twat
twats
twink
two
two's
twoo's
twoos.
twos!!
ugly
um
umm
ummm
ummmm
undies
undrgrndsound
university of oklahoma
unwed
up
update
upp
urinal
urine
uterus
uyuo
uzi
v1agra
va1jina
vag
vag1na
vagiina
vagina
vaginas
vaj1na
vajina
valium
viagra
victoria's secret
video watch
video!!!!
videoo
videooo
videoooo
videooooo
videos pls
virgin
vixen
vodka
vomit
voyeur
vulgar
vullva
vulva
vvideoo
vviideoo
w00se
w0p
wad
walkways
wang
wank
wanker
watt
wazoo
weathe
web
webpage
webpagee
website is messing up right now
websitee
wedgie
weeabo
weed
weenie
weewee
weiner
weirdo
wench
weree
wetback
wh00r
wh0re
wh0reface
whack titties
whack-off
whatt
whipping
white boy
whitey
whiz
whoar
whoralicious
whore
whorealicious
whored
whoreface
whorehopper
whorehouse
whores
whoring
wigger
womb
woody
wop
wowreallly
wtf
wtfiyp
wth
wut
www.bruinslist.com
www.bruinspokerclub.com
x-rated
xrated
xxx
xxxx
yabbo
yabbos
yahoo
yahoos
yaun
ybya
ybysa
yeasty
ykyarw
yobbo
you
zcgshpndpzruqjcyurch
zeeb
zigaboo
zoophile
 
Share