Data GIS Blog
What is Open Data?
Finding data can be challenging. Organizations and government agencies can share their data with the public using ESRI’s ArcGIS Open Data, a centralized spatial data clearinghouse. Since its inception last year, over 1,600 organizations have provided more than 22,000 open datasets to the public. Open Data allows users to find and download data in different formats, including shapefiles, spreadsheets, and KML documents, as well as APIs (GeoJSON or Esri GeoServices) to call the data into your own application. It also lets you create various types of charts.
How to Find and Use Data
Open Data allows consumers to type in a geographic area or a topic of interest in a single search box. Once you’ve found data that appears to be what you were looking for, you can use the data for GIS purposes or use a table to create charts and graphs. If you are looking for GIS data, you can preview the spatial data before downloading by clicking the “Open in ArcGIS” icon. This takes users to ArcGIS Online where they can create choropleth maps and interact with the attribute table. Users interested in tabular data can filter it and create various types of charts. If more analysis of the data is necessary, you can download it by clicking the “Download Dataset” icon; you are able to download the entire dataset or the filtered dataset you’ve been working with.
The Source and Metadata links below the “About” heading provide information about the data. In-depth information such as descriptions, attributes, and how the data was collected are provided in these links. Below the name of the dataset there are three tabs: “Details,” “Table,” and “Charts.” Under the “Details” tab there are three sections, the Description, Dataset Attributes, and Related Datasets sections. The Dataset Attributes section outlines the fields found within the dataset and provides field type information, while the Related Datasets section provides links to other datasets that have similar geographies or topics to the dataset you’ve chosen. In the “Table” tab, you can view and filter the entire table in the dataset and the “Charts” tab allows you to create different charts.
To obtain the most updated dataset or other updated articles related to the dataset, users should subscribe to the dataset they are interested in. To subscribe, copy the link provided into an RSS Reader. For specific data source questions, feel free to ask the Data and Visualization Department at firstname.lastname@example.org.
Our third year of the Duke Student Data Visualization Contest has come and gone, and we had another amazing group of submissions this year. The 19 visualizations submitted covered a very broad range of subject matter and visualization styles. Especially notable this year was the increase in use of graphic design software like Illustrator, Photoshop, and Inkscape to customize the design of the submissions. The winners and other submissions to the contest will soon be featured on the Duke Data Visualization Flickr Gallery.
As in the past, the submissions were judged on the basis of five criteria: insightfulness, broad appeal, aesthetics, technical merit, and novelty. The three winning submissions this year exemplify all of these and tell rich stories about three very different types of research projects. The winners will be honored at a public reception on Friday, April 10, from 2:00 p.m. to 3:00 p.m, in the Brandaleone Lab for Data and Visualization Services (in the Edge). They will each receive an Amazon gift card, and a poster version of the projects will be displayed in the lab. We are very grateful to Duke University Libraries and the Sanford School of Public Policy for sponsoring this year’s contest.First place:
Social Circles of Primary Caregivers / Tina Chen
Crystal Structure of Human Proliferating Cell Nuclear Antigen (PCNA) for in silico Drug Screen / Yuqian Shi
Deep and Extensive Impacts to Watershed Shape and Structure from Mountaintop Mining in West Virginia / Matthew Ross
Please join us in celebrating the outstanding work of these students, as well as the closing of the Places & Spaces: Mapping Science exhibit, on April 10 in the Edge.
Duke Libraries are happy to host the American Statistical Association’s Data Fest Competition the weekend of March 20-22nd. In its fourth year at Duke, DataFest brings teams of students from across the Research Triangle to compete in a weekend long competition that stresses data cleaning, analytics, and visualization skills. The Edge provides a central location for the competition with facilities designed for collaborative, data driven research.
While the deadline for forming DataFest teams has past, Data and Visualization Services and Duke’s Department of Statistical Sciences are happy to offer another opportunity to participate in DataFest. Starting Monday, March 16th we are offering four workshops on data analytics and visualization in the four days leading up to the DataFest event. All workshops are open to the public, but we strongly encourage early registration to ensure a seat. Please come join us as we get ready to celebrate ASA DataFest 2015.DataFest Workshop Series
Monday, March 16th, 6:00-8:00 PM – Introduction to R
Tuesday, March 17th, 1:30-3:00 PM – Easy Interactive Charts and Maps with Tableau
Wednesday, March 18th, 6:00-8:00 PM – Data Munging with R and dplyr
Thursday, March 19th, 7:00-9:00 PM – Visualization in d3
Last fall Duke University released its newest file sharing service known as Duke’s Box. By partnering with Box.com, Duke offers a cloud-storage service which is intuitive, secure, and easy to use. Login with with your NetID, share files with colleagues, and have confidence this cloud storage is compliant with all laws and regulations regarding data privacy and security.Simple to Use
Duke’s Box is similar to other cloud-based file storage services which support collaboration, productivity, and synchronization. You can drop and drag files, identify collaborators and set permissions (read, edit, comment, etc.) But unlike some services, such as Dropbox or Google Drive, Duke’s Box enables you to be in compliance with data privacy and security. Additionally, you can synchronize data across your devices, at your discretion and subject to Duke’s Security & Usage Practice restrictions
You may have used OIT’s NAS (Network Attached Storage) file storage service known as CIFS. Duke’s Box is easier to use although it provides services for slightly different use-cases. For example, CIFS might be more useful if accessing large files (e.g. video files that are larger than 5 GB). However, CIFS doesn’t enable collaboration or sharing. Depending on your needs you may still want to use your departmental or OIT NAS. Either way, you can use both file storage services and each service is free.
Check out this 5 minute quick-start video:50 GB of Space by Default
You are automatically provisioned 50 GB of space, but you can request more if you need more. See the FAQ for details.
Individual file size limitations are throttled to less than 5 GB. This means Duke’s Box may be less than ideal for sharing very large files. NAS services may be more appropriate for large files as the time to download or synchronize large files can become inconvenient. But for many common file sharing cases, Duke’s Box is ideal, fast and convenient.Documentation, Restrictions & Use
While you can store many types of files, there are best practices and restrictions you will want to review. For example, Duke Medicine users are required to complete an online training module prior to account activation.
- Security and Use, including more detail on Terms of Service, and example Data Types — including military and space data, FERPA, HIPAA, etc.
- Duke’s Box Usage Practices
- OIT’s FAQ
- Your Duke’s Box “Read Me” folder. OIT has done a great job of providing quick and convenient documentation located right where you need it. See the READ ME folder after you logon to Duke’s Box.
One of the many use-cases for Duke’s Box is a more convenient way for you to share your data with us. As you know we welcome questions about data analysis and visualization. We know describing data can be difficult while sharing your dataset can clarify your question. But sharing your data via email consumes a lot of resources — both yours and ours. Now there’s a better way; please share your data with us via Duke’s Box.Steps for Sharing Your Data with DVS Consultants
- Log into Duke’s Box (Use the blue “continue” button)
- Open your “home” folder
- Put your data in the “sharing” folder
- Use the “invite people” button (right-hand sidebar)
- Using a consultant email address, invite the DVS Consultant to see your data. (Don’t worry if you don’t have our email yet. When you start your question at askData@duke.edu, an individual consultant will be back in touch.)
This semester, Duke is proud to host the Places & Spaces: Mapping Science exhibit, visiting from Indiana University. Places & Spaces is a 10-year effort by Dr. Katy Börner (director of the Cyberinfrastructure for Network Science Center) to bring focus to visualization as a medium of scholarly communication.
The exhibit includes 100 maps from various disciplines and cultures and highlights myriad visualization techniques that have been used to communicate science to a broader public. The maps are divided among three spaces on campus: The Edge (newly opened on the first floor of Bostock Library), Smith Warehouse (on the second floor of Bay 11), and Gross Hall (on the third floor).
To celebrate the opening, Dr. Börner will visit Duke on January 21st and 22nd. She will give a keynote presentation on Wednesday, January 21, at 4pm, in the Edge. A reception will follow.
Additional events next week and throughout the semester will celebrate the exhibit and promote ongoing visualization work at Duke. All events are open to the public!Upcoming events
Wednesday, January 21
- Digital Scholarship Series opening session – Manifest Data, by the S-1 lab
3-4pm in the Edge
- Opening keynote by the exhibit’s curator, Dr. Katy Börner, and reception
4-6pm in the Edge
Thursday, January 22
- Screening of Dr Börner’s film Humanexus
12-1pm in 266 Smith Warehouse Bay 10
Friday, January 23
- Uncharted: Mapping the Spaces Between Disciplines (half-day conference)
9:45am-3pm in 217 Perkins
- Visualization Friday Forum – Exhibit tour in the Edge
12-1pm in the Edge
Please contact Angela Zoss (email@example.com) with any questions or suggestions. We hope you can join us in celebrating and enjoying this exhibit!
Data and Visualization Services is happy to announce our new Data and Visualization Lab in Duke Libraries new Edge research space. Located on the first floor of the Bostock Library, the Brandaleone Family Lab for Data and Visualization Services offers a dedicated space for researchers working on data driven projects.
The lab features three distinct areas for supporting data driven research.Data and Visualization Lab Space
Our lab space features twelve high end workstations with dual monitors with the latest software for data visualization, digital mapping, statistics, and qualitative research. All of the machines have two dedicated displays to encourage collaborative work and data consultations. Additionally, all twelve machines have a dedicated power port located conveniently under the edge of the table for powering a laptop or usb powered device.Bloomberg Professional “Bar”
Since the launch of our Bloomberg terminals, we have seen a steady increase in both individual and team based usage of Bloomberg financial data. Our three Bloomberg Professional workstations are now located on a dedicated “bar” across from our lab machines. The new Bloomberg zone will facilitate collaborate work and provide a base for groups such as the Duke University Investment Club and Duke Financial Economics Center.Consult and Collaborative Space
Our third lab space provides a set of four rolling tables for small groups to collaborate or for projects that don’t require a fixed computing space. An 85″ flat panel display near this zone features data visualizations and other data driven research projects at Duke.Come See Us!
With ample natural light, almost 24/7 availability, and a welcoming staff eager to work with you on your next data driven project. We look forward to working with you in the upcoming year!