More details on the map and its methodology from the research team:

The data behind this map is based on every geocoded tweet in the United States from June 2012 – April 2013 containing one of the ‘hate words’. This equated to over 150,000 tweets and was drawn from the  DOLLY project based at the University of Kentucky. Because algorithmic sentiment analysis would automatically classify any tweet containing ‘hate words’ as “negative,” this project relied upon the HSU students to read the entirety of tweet and classify it as positive, neutral or negative based on a predefined rubric. Only those tweets that were identified by human readers as negative were used in this analysis.

To produce the map all tweets containing each ‘hate word’ were aggregated to the county level and normalized by the total twitter traffic in each county. Counties were reduced to their centroids and assigned a weight derived from this normalization process. This was used to generate a heat map that demonstrates the variability in the frequency of hateful tweets relative to all tweets over space. Where there is a larger proportion of negative tweets referencing a particular ‘hate word’ the region appears red on the map, where the proportion is moderate, the word was used less (although still more than the national average) and appears a pale blue on the map. Areas without shading indicate places that have a lower proportion of negative tweets relative to the national average.

The numbers that appear in the map during a mouse hover indicate the total number of hateful tweets and number of unique users sending them in each county.

It’s a sad state of affairs that hateful words still run rampant in this country, but perhaps by isolating “hotspots” of hate, we’re one step closer to educating our way into being a country where tolerance rules the day.

—

The Geography of Hate is part of a larger project by Dr. Monica Stephens of Humboldt State University (HSU) identifying the geographic origins of online hate speech. Undergraduate students Amelia Egle, Matthew Eiben and Miles Ross, worked to produce the data and this map as part of Dr. Stephens’ Advanced Cartography course at Humboldt State University. Read more about the research and methods behind this project at www.FloatingSheep.org.

Twitter data was obtained from the DOLLY project at University of Kentucky, and the map was built on the Google Maps API.

via Bay McLaughlin (@betabay)

The post Geography of Hate: Geotagged Hateful Tweets in the United States appeared first on Visualizing Systems.

http://drones.pitchinteractive.com/

The post Out of Sight, Out of Mind: Drone Attacks in Pakistan since 2002 (Pitch Interactive) appeared first on Visualizing Systems.

Many of you probably read the New York Times feature last year describing how Target used its data collection team to discover shoppers who were pregnant—and therefore likely to become big spenders on diapers, lotions, baby furniture, and by extension, everything else if they could be targeted (no pun intended) early enough in the pregnancy. Citing the poignant example of an enraged father deploring Target for suggesting that his teenage daughter might be pregnant, only to discover that she actually WAS pregnant, this article raised the question, when does data collection cross the line between smart business practice and invasion of privacy?

A recent mapping project, the Internet Census 2012, blurs the line even further. Hot on the tails on SOPA, PIPA, and Aaron Swartz’s untimely death, and the conversation about open access to information these events provoked, this project by an anonymous researcher capitalized on a glaring security loophole in Linux systems to find temporary hosts that would allow a good chunk of the IPv4 internet (note 1) to be mapped.

The technical details are beyond the scope of this blog and are described in further detail in the report, but essentially, the researcher hacked into 420,000 Linux systems that had either weak or no system passwords,  infecting each device with a background service that scanned for IP addresses. Luckily, the researcher harbored no malicious intent: the system was not affected, incoming and outgoing traffic were ignored, and each device was returned to its original state after a reboot.

The maps and visualizations created as a result of this endeavor are stunning for their revelations about the internet as much as for the chutzpah involved in embarking on such an ethically-ambiguous endeavor. Is it worth the potential violations of privacy of hundreds of thousands of individuals (and corporations) to document the proliferation of our virtual communications networks? How far is it prudent to go in the pursuit of knowledge acquisition, and where do we draw the line between information we NEED and information we WANT? After all, the creator of this work was cited as saying, “I did not want to ask myself for the rest of my life how much fun it could have been or if the infrastructure I imagined in my head would have worked as expected.”

True, the hacker involved had the ethical bearings to restrain this study, and did not exploit the private contents of the devices and local networks, but what about next time? In all the project charted 4.6 million IP addresses of the course of October to December 2012, affecting hundreds of thousands of devices and in the process breaking enough laws around the world to “make them liable for many thousands of years behind bars” if current sentencing policy prevails. A big if, considering recent forays into information access, whether legal (Freedom of Information Act (FOIA)) or not (WikiLeaks).

This post is by no means critical of the results, either of Target or the Internet Census, as obviously data mapping is the prime motivator of this blog. Indeed, the maps and diagrams themselves give a clear picture of the concentration of the internet at large in a way that more abstract maps by CAIDA, Opte, and others (note 2) are unable to do. Given the optimistic, save-the-world attitude often embraced by mappers, data visualizers, and urban explorers such as myself, it is merely worth asking, how much data do we need? And how far are we willing to go to get it

1. World map showing the entire Carna Botnet client distribution, from March to December 2012 – each dot represents between 100 and 2000+ botnet clients, for a total of 420,000

2. World map showing 4.6 million IPv4 addresses mapped between October and December 2012.

To get a geographic overview we determined the geolocation of all IP addresses that respond to ICMP ping requests or have open ports. We used MaxMinds freely available GeoLite database [maxmind.com] for geolocation mapping. Different versions of this image are available for download at http://internetcensus2012.bitbucket.org/images.html

3. Relative IPv4 utilization observed using ICMP Ping requests – World Map

To test if we could see a day night rhythm in the utilization of IP spaces we used all ICMP records to generate a series of images that show the difference from daily average utilization per half an hour. We composed theses images to a GIF animation that clearly shows a day night rhythm. The difference between day and night is lower for US and Central Europe because of the higher number of “always on” Internet connections.

4. Relative IPv4 utilization observed using ICMP Ping requests – Hilbert Diagram

To get a visual overview of ICMP records we converted the one-dimensional, 32-bit IP addresses into two dimensions using a Hilbert Curve, inspired by xkcd. This curve keeps nearby addresses physically near each other and it is fractal, so we can zoom in or out to control detail. Figure 2 shows 420 Million IP addresses that responded to ICMP ping requests at least two times between June and October 2012. Address blocks are labeled based on IANA’s list of IPv4 allocations that can be found here. Each pixel in the original 4096 x 4096 image represents a single /24 network containing up to 256 hosts. The pixel color shows the utilization of each /24 based on the number of probe responses. Black areas represent addresses that did not respond to the probes. Blue represents low utilization (at least one response), and red represents 100% utilization. This image was generated to be comparable to Figure 3, created 2006 by CAIDA in an Internet census project [isi.edu].

5. Static Hilbert Map of the IPv4 internet

(note 1) IPv4 is the system of establishing a unique number to identify devices on a network. If you’ve ever seen a number like 192.168.1.42, that’s an IP4 address. IPv4 is a 32-bit system, meaning the address format can provide 4.29 billion (2^32) different configurations–and when the internet was in its infancy, this seemed like a LOT of devices. However, the last blocks of IPv4 addresses were officially allocated in February 2011. Luckily, a switch had been in the works for several years to IPv6, a 128-bit system that allows for 2^128 Internet addresses, or 340,282,366,920,938,000,000,000,000,000,000,000,000 of them to be exact. For more information on this switch, Mashable has a good plain-English explanation.

(note 2) For more maps of the internet and other virtual systems, see http://visualizingsystems.com/category/connective-systems/virtual-infrastructure/

via @KaiStinchcombe

The post On the Ethics of Mapping: When does data collection cross the line? appeared first on Visualizing Systems.

What’s more telling about these maps though, whether you live in London or not, is the way they they visually highlight the predominant source of the problem: “Much of the pollution in London,” the researchers write, “is associated with traffic.”

We’re essentially looking at the city’s street network, with the busiest roads also doubling as the worst sources of pollution. And that pattern would appear the same in cities everywhere. All of the pollution levels currently shown are considered low and unlikely to cause health effects. But these pictures give a good indication of where we might want to target efforts to reduce pollution even further.

The post ‘Nowcasting’ Pollution on the Streets of London appeared first on Visualizing Systems.

According to science, love really is a state of mind, a kind of drunkenness induced by a complex yet primitive neurochemical cocktail–the brain being one kick-ass mixologist. Scientific American hones in on love in the brain, highlighting the exact locations where different stages of love (or at least the chemical reactions they produce) take place. There are about a dozen centers of love in the brain, which differ depending on whether one is experiencing passionate love, maternal love, or unconditional love. It also breaks down such abstract feelings as trust and attraction into their requisite neurotransmitters. So just remember, tonight when you are experiencing a a flurry of attraction or woozy butterflies that accompany your candlelit dinner date, you can chalk it up to dopamine or vasopressin lighting up those passionate regions of your Dorsolateral middle frontal gyrus. Romantic, eh?

In the spirit of primal love, the internet has made one Valentine’s Day tradition just a little bit easier. Now, coupled-up individuals can flaunt their status to hapless singletons digitally, thanks to IJustMadeLove.com. Replete with humping bunnies, this site takes Google Maps API-crawling to a whole new level, allowing users from all over the world to brag (anonymously) about their sexual accomplishments. I’ll leave it to you to discover all of the additional features, but suffice it to say this is a prime example of data-rich, crowd-sourced, real-time mapping if ever there was one.

I Just Made Love

The post Mapping Love (and everything else that comes with it) appeared first on Visualizing Systems.

VISUALIZING SYSTEMS: TIMELINE OF HISTORICAL VISUALIZATION

Visualizing Systems presents an in-depth timeline featuring many of the most important visualizations and mappings of the human environment from 1800 – present. The collection considers the three-dimensional world—and the spatial, material, and temporal complexities that come with it—in order to document visual language that balances quantitative rigor and scientific methodology with tangible experiential descriptions of the groundplane as seen at eye level.

The exhibit aims to harness growing interest in data visualization and shift its focus toward the spatial environment. Moreover, through the cultivation of a rich collection of historic imagery, the exhibit and site shine a light upon the origins of technologies and techniques that are all but ubiquitous today–while infographics and data visualizations are trending now, however, the methods they employ are founded upon several milestones in visualization and analysis—from design, art, and the sciences—that are called out in this exhibit. The experiments section of the exhibit, below, retraces the development of representational techniques that are commonplace today so that we may encode new layers of meaning, thereby increasing the operativity of the drawing.

VISUALIZING SYSTEMS: THE EXPERIMENTS

The post Visualizing Systems Exhibit at Harvard GSD appeared first on Visualizing Systems.

The collection considers the three-dimensional world—and the spatial, material, and temporal complexities that come with it—in order to document visual language that balances quantitative rigor and scientific methodology with tangible experiential descriptions of the groundplane as seen at eye level. What are the relationships between cities and their ecological contexts? How can we harness the increasing power and availability of big data to map and track the relationships between urban and ecological systems, nature and culture, and landscape and infrastructure, in a way that is not only BEAUTIFUL, but also PRECISE and CONCISE? And most importantly, how can the creation of clear and concise visualizations that explain a complex and changing world help to confer a new legitimacy to architects, landscape architects, and planners as they aim to carve out more prominent roles as policy makers?

After the talk we had a great discussion about the role of representation in landscape architecture, and how to challenge landscape architecture practice to make better use of data. How can we make better use of mobile technology, iPads and iPhones, etc. to show our drawings in motion? What kinds of information can we extract from historic maps and drawings, and how does this inform contemporary work?

Thanks to all who were able to attend the talk, and for the many questions and suggestions to improve this blog and book moving forward.

For more information about the Kiley Fellowship, visit http://www.gsd.harvard.edu/#/academic-programs/landscape-architecture/kiley-teaching-fellowship.html

The post Lecture on VIS SYS at Harvard GSD appeared first on Visualizing Systems.

Let’s start off with the simplest and then get more complex. First, I would be remiss if I didn’t mention this year’s election data darling, Nate Silver of FiveThirtyEight, a blog which the New York Times jumped on in 2010 after Silver’s free-standing blog correctly predicted 49 of 50 states in the 2008 election. The graphics used by FiveThirtyEight are generally extremely lightweight, aiming for clarity and immediate legibility over excitement: all are excellent examples of Edward Tufte’s principles of avoiding chartjunk and maintaining a high data-ink ratio. I particularly like the following chart for its incorporation of sparklines, another Tufte invention, to show the trend in swing-state polls across a series of timespans: 1) the period between Romney becoming the official GOP nominee (6/1) and the end of both party conventions (9/27), 2) the period between conventions and the Denver debate (10/4), 3) the time between the Denver debate and the second debate (10/13), 4) the time between the second and third debates, and 5) the third debate until the date the chart was published (11/1). While the use of subtle blue and red shading gives a sense of the overall trends, the sparklines give an immediate reading of the ups and downs of the poll averages in each state.

Let’s look now at a few more graphics from the New York Times, who are a very consistent source of infographics of the clear and simple variety. One graphic that has gotten a lot of attention recently is a fantastic interactive graphic by Mike Bostok and Shan Carter that charts, through electoral vote count, how the states have shifted party affiliation over every presidential election since 1952. I love the way this graphic conveys so much data so simply. By my count, there are five variables shown: 1) time (Y-axis), 2) percent of vote (X-axis), 3/4) number of electoral votes and percentage of total (thickness of line), and 5) winning party (red or blue color). This is a supergraphic in every sense of the word: it is dense, content-rich, and equally compelling at a macro and micro level. It’s the kind of graphic that you can read in a few seconds, or spend an hour playing around with and making new discoveries.

Another great interactive graphic by Bostock and Carter just published today uses a tree structure to examine all of the possible roads to victory (270 electoral votes) for Barack Obama and Mitt Romney. This is another graphic that you can spend hours playing with, testing out different scenarios, and really, the graphic is best used for that purpose. Where it is less successful is in conveying the most likely outcomes visually. While right now the line width is representative of the number of electoral votes each branch produces, perhaps at least the option of a different weighting that reflected probabilities would be useful.

The post Election Infographics: The top 1% of visualizations (and the bottom 47%) appeared first on Visualizing Systems.

Let’s start with the sheer scale of these storms. One thing that is difficult to sense on the ground is the awe-inspiring expanse of a tropical storm, Sandy being even more gargantuan than usual. Indeed, the scale of these storms both magnifies the impending doom and diminishes it: unlike the tornados and supercell thunderstorms I recall from my childhood in Oklahoma, mile-wide storms which emerge in seconds and move like oncoming freight train, storms such as Sandy and Irene are hundreds of miles wide and move slowly–about 20 miles an hour at their maximum speeds. The Wall Street Journal posted a great interactive satellite photo comparison of Hurricanes Irene and Sandy that captures what this means: while Irene is huge, Sandy is a real monster, literally stretching along the east coast.

Left: NOAA/Associated Press; Right: NASA/Getty Images

Perhaps most revealing of the gargantuan scale of these storms, however, is a series of fake photos that went viral today, in which images of supercell thunderstorms were convincingly photoshopped onto New York landmarks. While any educated observer (or blogger raised in Oklahoma) might instantly smell a fake, these photos succeed where perhaps photos of the real thing don’t: by reducing the terrifying, churning storm to the scale of the city, the photo is able to instantly convey a sense of impending doom that photos of gray skies and windswept trees–the realistically-scaled atmospheric effects–can’t quite match.

The post The Eye of the Storm: Visualizing Weather appeared first on Visualizing Systems.

It is interesting to compare the evacuation map to a 1776 map of the island before much of the coastline was augmented by landfill. The eastern line of Zone A along the Hudson River runs along Greenwich Street, which was at the waterfront in 1776. The old slips on the East River extend inland to Queen Street, now Pearl Street, which is near where Zone A runs along the East River.

Also notable on the 1776 map is Bayard’s Mount, the high land rising in the area marked “Marshy Ground”  north and northwest of the old Collect Pond. The pond was drained in the early 19th Century and Bayard’s Mount was leveled to fill it in, but as can be seen in the evacuation plan, the pond and the marsh left their mark on modern Manhattan in the form of a hook-shaped low area delineated by the border of Zone B.

* The NYC Sandy evacuation plan is a great model of rapid digital deployment which Fast.Co.Design describes in some detail: “The street grid comes from OpenStreetMap, the free, editable map of the world based on open source software. The zones and shelter data were overlaid using MapBox, a fast way to create and publish your own maps with an open API. The map builds on the success of the city’s Hurricane Irene map. NYC’s Chief Digital Officer Rachel Sterne explains that she wants to “encourage expert designers and developers to take advantage of the geographic files corresponding to hurricane evacuation zones, available free of charge on the NYC Open Data platform.” More info on NYC’s other digital initiatives at http://fop.good.is/figures/rachel-sterne.

**Take a look at the Kreilsheimer Pavilion by Kathryn Gustafson for this in action at a micro scale.

The post The Agency of the Contour: Evacuation plan reveals historic Manhattan waterline appeared first on Visualizing Systems.