13 Dec 2018: "Developing JupyterLab Extensions": Ian Rose (Berkeley)

Summary: JupyterLab extensions can customize or enhance any part of JupyterLab. They can provide new themes, file viewers and editors, or renderers for rich outputs in notebooks. Extensions can add items to the menu or command palette, keyboard shortcuts, or settings in the settings system. Extensions can provide an API for other extensions to use and can depend on other extensions. In fact, the whole of JupyterLab itself is simply a collection of extensions that are no more powerful or privileged than any custom extension. In this talk Ian will demonstrate how to build a JuptyerLab extension.

Summary: Intake is a set of free open-source Python tools that help load data from a variety of formats into familiar containers like Pandas dataframes, Xarray datasets, and more. Boilerplate data loading code can be transformed into reusable Intake plugins. Datasets can be described for easy reuse and sharing using Intake catalog files. Martin will give an overview of Intake and demonstrate use via Jupyter Notebooks.

Recording

Summary: Building a National Biogeographic Map, an analysis platform for exploring biodiversity conservation measures and stressors, sparked the need for a reliable data source with a wide variety of identified places. These need to be assembled in a sustainable and robust way that keeps track of provenance and processing steps so we can build reports for decisionmakers that are trustworthy and consistent. We started an idea campaign in the ESIP Lab with some questions we have about how best to do this work. This talk will share what we have in place so far, including a registry of sources, a data processing pipeline, an integrated index, a REST API, and a working web application. Technologies include USGS ScienceBase, PostgreSQL/PostGIS, ElasticSearch, Python Flask, and other Python processing codes using GDAL and other libraries.

Join meeting:
No live meeting, as something came up at the last minute and Sky can't make it. He did record a 30 minute presentation, however. See "Recording" below. See also the sample Jupyter notebooks in the "Links" section below.

Speaker: Sky Bristol is the branch chief for Biogeographic Characterization in the USGS Core Science Analytics, Synthesis, and Library Program. Unless he can be skiing bumps, he likes doing cool things to help people make better decisions using data.

Recording

Summary: Building environmental simulation workflows is typically a slow process involving multiple proprietary desktop tools that do not interoperate well. In this work, we demonstrate building flexible, lightweight workflows entirely in Jupyter notebooks. The goal is to provide a set of tools that can easily be reconfigured and repurposed as needed to rapidly solve specific emerging issues. As part of this work, extensive improvements were made to several general-purpose open source packages, including support for annotating and editing plots and maps in Bokeh and HoloViews, rendering large triangular meshes and regridding large raster data in HoloViews, GeoViews, and Datashader, and widget libraries for Param.

Speaker:
Dharhas Pothina is the Associate Technical Director of the Information Technology Laboratory, US Army Engineer Research and Development Center, in Vicksburg, MS. He was formerly the Water Informatics Lead at the Texas Water Development Board. He holds a Ph.D in Civil Engineering from the University of Texas at Austin.

Recording

Summary: An open-source environment for parallel analysis of massive (100TB) image data in the Cloud is now available via the Pangeo environment, which allows you to apply the power of the Python ecosystem from your browser. Technologies include JupyterHub, Kubernetes, Docker, and Dask distributed.

Speaker(s):
Tim Crone is a marine geophysicist at Lamont-Doherty Earth Observatory studying spatial variations in the tidal triggering of microearthquakes within ridge systems, and problems in acoustics associated with high-temperature hydrothermal vents and seafloor seismic networks. He has recently deployed the Pangeo framework on the Microsoft Azure Cloud.

Recording

Summary: Simple geometry (points, lines, and polygons) has now been accepted as part of the Open Geospatial Consortium’s NetCDF-CF specification. This a major enhancement to a widely used standard whose utility has previously been limited to time-series of point or (raster) coverage data only. Advances on Groups and Swaths will also be presented.

Speaker(s):
Dave Blodgett is a project coordinator with the USGS Office of Water, Geointelligence Branch. He holds a B.S. in civil and environmental engineering and an M.S. in water resources engineering form the UW-Madison. Tim Whiteaker is a research scientist at the University of Texas at Austin. Tim develops innovative cyberinfrastructure for solving water resource engineering challenges. Aleksander Jelanek is a Senior Informatics Architect with the HDF Group, and Daniel Lee is a Software Engineer at EUMETSAT.

Recording

Summary: Jetstream, a national science and engineering cloud, adds cloud-based, on-demand computing and data analysis resources to the national XSEDE cyberinfrastructure. A description of Jetstream current and planned capabilities, and how to gain access, will be presented.

Speaker(s):
Jeremy Fischer is a Senior Technical Advisor at Indiana University. He works primarily on the Jetstream project, as the technical evangelist getting researchers and educators on the system. In this role, he is the jack of all trades doing unix sys admin work, cloud image maintenance, support, training, documentation, and anything else that needs to happen.

Speaker(s):
Alistair Miles is the Head of Epidemiological Informatics
for the Kwiatkowski group at the University of Oxford. Before joining
the University of Oxford, Alistair was a research scientist at the
e-Science Centre at the UK Science and Technology Facilities Council,
where he was involved in a range of computing research projects,
primarily in the areas of Web and semantic technology, and also in the
engineering of production software systems. He is the lead developer
for Zarr.

Recording

Summary: The National Data Service Labs Workbench is a platform designed to share, discover, evaluate, develop, and test research data management and analysis tools. Community members can recommend or contribute tools as well as drive the direction, and the Workbench is evolving into a platform for data access, education and training.

Speaker(s):
Craig Willis is the Technical Coordinator for the National Data Service and a senior research programmer at the National Center for Supercomputer Applications (NCSA) at the University of Illinois.

Recording

Summary: Pangeo is a scalable, low-barrier-for-entry science platform, with cloud-optimized storage for large multidimensional datasets, such as simulation (met, ocean, hydrologic, climate) model output. Technologies include JupyterHub, Kubernetes, Xarray, Dask, and Zarr. The Pangeo environment has been deployed on the NCAR Cheyenne supercomputer, on Google Cloud and on AWS.

Speaker(s):
Ryan Abernathy is a physical oceanographer at Lamont/Columbia, and Matthew Rocklin is a open-source developer for Anaconda.

Recording

Summary: Jupyter widgets (aka ipywidgets) enables building interactive GUIs for Python code using standard form controls (sliders, dropdowns, textboxes, etc.), as well providing a framework for building complex interactive controls such as interactive 2d graphs, 3d graphics, maps, and more. Jason will walk through the thought and technical processes involved with developing new widget capability.

Speaker(s):
Jason Grout is scientific software developer at Bloomberg. He has been a member of the Project Jupyter team since it's inception in 2014 and a core developer for the Jupyter widgets project. He has a PhD in mathematics from Brigham Young University.

Recording

9 November 2017: "Jupyter Widgets": Jason Grout, Bloomberg

Summary: Jupyter widgets (aka ipywidgets) enables building interactive GUIs for Python code using standard form controls (sliders, dropdowns, textboxes, etc.), as well providing a framework for building complex interactive controls such as interactive 2d graphs, 3d graphics, maps, and more. The latest developments in Jupyter widgets will be discussed as well as plans for the future.

Speaker(s):
Jason Grout is scientific software developer at Bloomberg. He has been a member of the Project Jupyter team since it's inception in 2014 and a core developer for the Jupyter widgets project. He has a PhD in mathematics from Brigham Young University.

Recording

Summary: The Research Workspace (RW) is a web-based tool designed to support collaborative science and data management tasks throughout the data lifecycle. The RW provides a secure environment for organizing, sharing, documenting, and analyzing scientific datasets, and for publishing datasets through a DataONE member node. As a shared, cloud-based storage environment, the RW is designed for collaborative organization and management of project content. Multiple levels of access and read/write permissions provide transparent, controlled access and oversight to collaborators, funders, and project managers. A custom metadata editor exports standards-compliant metadata (ISO 19115-2 and 19110) and includes tools for easily adding keywords, taxonomic information, keywords, spatial boundaries, and contact information to metadata records. An integrated Jupyter notebooks environment allows R- and Python-based analysis scripts to be written in and run on the RW and to access data in the RW or any data set resource that is hosted within the Axiom Data Science cyber-infrastructure stack. These notebooks serve as transparent, reproducible, and easily-shareable computational analysis and processing tools. Finally, datasets in the RW that have undergone sufficient curation and documentation can be exported to the Research Workspace DataONE Member Node for long-term preservation and broader discoverability and accessibility. More information about the RW and it's capabilities can be found in the help documents -

Speaker(s):
Rob Bochenek is an information architect at Axiom Data Science. Rob has been developing data management and cyber infrastructure solutions for research programs and organizations for the past fifteen years. He is a graduate of the University of Michigan with a degrees in aerospace engineering and mathematics. Early in his career Rob spent five years at the Exxon Valdez Oil Spill Trustee Council leading the data management team in processing, documenting and organizing the informational products produced from the scientific research funded to understand and monitor the ecological effects of the oil spill. Based upon that experience, Rob founded Axiom in 2006 to develop more generalized and holistic solutions for data management. He specializes in scientific geospatial information management with applications to physical/biological modeling and decision support data warehouse knowledge systems.

Speaker(s):
Brian Granger is an Associate Professor of Physics at Cal Poly State University in San Luis Obispo, CA. He has a background in theoretical atomic, molecular and optical physics, with a PhD from the University of Colorado. His current research interests include quantum computing, parallel and distributed computing and interactive computing environments for scientific and technical computing. He is a core developer of the Jupyter project and is an active contributor to a number of other open source projects focused on scientific computing in Python.

Recording

Summary: ERDDAP is a free, open source data server that gives you a simple, consistent way to download subsets of gridded and tabular scientific datasets in common file formats and make graphs and maps.

Speaker(s):
Yuvi Panda is a developer with 15 years of experience and 400+ followers on GitHub. He worked formerly with Wikimedia, and is currently working with the Data Science Education Program at UC Berkeley to make it easier for people who don't consider
themselves programmers to write code. He has been very involved with creating the Helm Chart for JupyterHub.

Kevin Ring is a Principal Software Engineer at CSIRO's Data61, and is the lead developer for TerriaJS. Previously, he helped found the Cesium project while working at Analytical Graphics, Inc. (AGI) and developed its streaming terrain and imagery engine.

Recording

13 April 2017: "Processing Planetary-Scale Data in the Cloud": Drew Bollinger, Development Seed

Summary: Modern cloud-based infrastructure has had a huge effect on our ability to process, manipulate, and publish satellite imagery at scale. We'll discuss current methods of making imagery available across different platforms and how this is supported by the efforts of groups like AWS to publish open satellite data including MODIS, Landsat and more.

Drew Bollinger is a data analyst and software developer, with experience running advanced statistical and spatial analysis on large and small data sets, as well as building visualizations for data storytelling.

Slides

Recording

Summary: Today, multi-media communication plays a pivotal role in how an audience experiences, understands, and shares your message. Story Maps bring a narrative to life by weaving maps, text, images, video, and other content into a creative and memorable story. Christine will share several examples of effective Story Maps and then walk through how you can create and configure your own.

Christine is a Technical Advisor and science team member at Esri. She loves using art and technology to communicate about the challenges and opportunities for our future. Christine also serves as the Vice President of ESIP. One of her favorite things about ESIP is how its members offer their unique perspectives (stories) and shared knowledge to collaborate.

GoToMeeting Recording

Slides

9 February 2017: "Web AppBuilder for ArcGIS": Derek Law, ESRI

Summary: Web AppBuilder for ArcGIS is a pure HTML5/JavaScript-based application that allows you to create your own intuitive, fast, and beautiful web apps without writing a single line of code. The app uses new ArcGIS platform features and modern browser technology to provide both flexible and powerful capabilities such as 3D visualization of data. In addition, developers have an opportunity to create custom tools and themes through the extensibility framework.

Jessica Walker is a postdoctoral researcher with the USGS Western Geographic Science Center in Tucson, AZ. Her research investigates the recovery of post-wildfire landscapes in Alaska and across the southwestern US using time series of remote sensing imagery.

GoToMeeting Recording

Slides

8 December 2016: "Vector Tile Maps": Sam Matthews, Mapbox

Summary: Vector tiles make huge maps fast while offering full design flexibility. They are the vector data equivalent of image tiles for web mapping, applying the strengths of tiling – developed for caching, scaling and serving map imagery rapidly – to vector data. A general overview of vector tiles will be presented.

Speaker(s):

Sam Matthews is a Mapbox engineer focused on improving the speed and reliability of maps. He works with the Mapnik team to generate vector tiles and maintains the upload pipeline behind Mapbox Studio. He is passionate about making open source tools as welcoming as possible through clear docs and zero assumptions.

GoToMeeting Recording

Slides

10 November 2016: "Introducing 3D Tiles": Todd Smith, AGI

Summary: 3D Tiles are an open specification for streaming massive heterogeneous 3D geospatial datasets. To expand on Cesium’s terrain and imagery streaming, 3D Tiles will be used to stream 3D content, including buildings, trees, point clouds, and vector data.

Speaker(s):

Todd Smith is the Cesium Product Manager, and helps define and manage the Cesium product line. Todd has been with the AGI team from the beginning and has been in the web mapping world for over 15 years. He is a Penn State GIS graduate.

GoToMeeting Recording

Slides

Summary: An outgrowth of activities of the EarthCube Technology Architecture Committee (TAC)'s Testbed Working Group (TWG), ECITE provides an integration test-bed for technology and science projects for both EarthCube funded projects and community technology demonstrations. ECITE consists of a seamless federated system of scalable and location independent distributed computational resources (nodes) across the US. The hybrid federated system provides a robust set of distributed resources utilizing including both public and private cloud capabilities.

Speaker(s): Chaowei Phil Yang is a Professor at George Mason University where he founded the NSF Spatiotemporal Innovation Center with colleagues from Harvard and UC-Santa Barbara. He advised over 30 graduate students and has placed over 20 geoinformatics professors around the world. His research interest are utilizing spatiotemporal principles to optimize computing infrastructure for geospatial science applications of national and international significance. (http://cpgis.gmu.edu/homepage/)

Speaker(s): Lewis McGibbney, NASA JPL/Apache OCW; currently a Data Scientist at the NASA Jet Propulsion Laboratory in Pasadena, California, Lewis works in the Computer Science and Data Intensive Applications Group (398M). He enjoys floating up and down the tide of technologies at the Apache Software Foundation having a real enthusiasm for Web Search and Information Retrieval in particular. You'll find him on community mailing lists including Nutch, Gora, Any23, OODT, Open Climate Workbench, Tika, Usergrid and a number of incubating mailing lists including CommonsRDF, HTrace and Joshua. Lewis is currently a Project Management Committee member and Committer on OCW.

Speaker(s): Huikyo Lee, NASA JPL/Apache OCW; currently a Climate Data Scientist at the NASA Jet Propulsion Laboratory in Pasadena, California, Huikyo has lead development of Regional Climate Model Evaluation System (http://rcmes.jpl.nasa.gov), an open-source software toolkit based on Open Climate Workbench to facilitate systematic evaluation of climate models using observational datasets from a variety of sources.

GoToMeeting Recording

Slides

Summary: CDAT is a rich set of visual-data exploration and analysis capabilities well-suited for earth science data analysis problems. It integrates many tools and technology to offer scientist a start-to-finish environment for their work. From reading in various data format, to publication-quality output of their analysis.

Speaker: Charles Doutriaux is a senior Lawrence Livermore National Laboratory research computer scientist, where he is known for his work in climate analytics, informatics, and management systems supporting model intercomparison projects. He works closely with many international climate scientists and shares in the recognition of the Intergovernmental Panel on Climate Change 2007 Nobel Peace Prize. He has co­-authored over 30 peer­-reviewed articles. He presented his work to many scientific conferences. Aside from everything Python-related, his research interests include climate attribution and detection, visualization, and data analysis. Doutriaux has a master's degree in "Climate and Physico-­Chemistry of the Atmosphere" from the University Joseph Fourier in Grenoble. He’s a member of the AGU and AMS. You can contact him at doutriaux1@llnl.gov.

GoToMeeting Recording

Slides

Summary: The OneStop Project is designed to improve NOAA's data discovery and access framework. Focusing on all layers of the framework and not just the user interface, OneStop is addressing data format and metadata best practices, ensuring more data are available through modern web services, working to improve the relevance of dataset searches, and improving both collection-level metadata management and granule level metadata systems to accommodate the wide variety and vast scale of NOAA's data.

Speaker: Ken Casey is the Deputy Director of the Data Stewardship Division in the NOAA National Centers for Environmental Information (NCEI). He leads the OneStop project, is active within NOAA's Big Earth Data Initiative and Big Data Project. Ken serves on a variety of national and international science and data management panels including the US Group on Earth Observations Data Management Working Group and the Group for High Resolution Sea Surface Temperature (GHRSST) Science Team. He co-chairs the Committee on Earth Observing Satellites SST Virtual Constellation and represents NCEI in the Federation of Earth Science Information Partners (ESIP). He holds a PhD in Physical Oceanography from the University of Rhode Island.

GoToMeeting Recording

Slides

Summary: Docker is an open platform for distributed applications that has taken the world by storm, making it easy to deploy services with complicated dependencies. In this presentation you will learn what Docker is, why it will make your life easier, how to build a container, and how to install containers.

Speaker: Kyle Wilcox, Dave Foster and Shane StClair are developers at Axiom Data Science. Axiom Data Science works with organizations to improve the long term management, reuse and impact of their scientific data resources. They have built Docker containers for many of the key services used by the U.S. Integrated Ocean Observing System (US-IOOS).

GoToMeeting Recording

Slides

12 May 2016: "Leaflet Time Dimension": Biel Frontera, SOCIB

Summary: Leaflet.TimeDimension is a free, open-source Leaflet.js plugin that enables visualization of spatial data with a temporal dimension. It can manage different types of layers (WMS, GeoJSON, Overlay) and it can be easily extended. It meet some common needs, enabling web maps using observational and forecasting layers generated by a THREDDS server (via ncWMS), animating trajectories of drifters, gliders, follow a simulated oil spill, and other time dependent mapping applications.

Speaker: Biel Frontera was trained as a mathematician, and has spent most of his career developing software. He is a free software enthusiast and has worked for the last 3 years on data visualization and geospatial software issues for SOCIB, the Baleric Islands Coastal Observing and Forecasting System.

GoToMeeting Recording

Slides

The Thriving Earth Exchange is a network and platform that connects community leaders, sponsors, and scientists and helps them combine science and local knowledge to solve on-the-ground challenges related to natural hazards, natural resources, and climate change. I’ll talk about the general principles on which we are building TEX and describe the basic modules that are part of the TEX. Drawing on the lessons learned from our pilots, I'll talk about how we are developing modules and launching new projects with several partners. I’ll describe a range of projects – from a community monitoring effort in Denver to a Pamiri Mountain project to integrate climate projections into traditional calendars. I’ll introduce our nascent “share” module, and describe our partnership with Amazon Web Services to move prototype community-based solutions to the cloud to enhance their adaptability. And, just to live up to the name, I’ll frame it all around a small rant about the loading-dock model of science and a rave about more participatory approaches.

This presentation will demonstrate how to use this system in a
reproducible Jupyter Notebook, discovering, accessing and using model
and observed water levels along the US Coastline, using a free python
environment that can be installed on Mac, Windows and Linux in less
than 10 minutes.

Slides

WebEx Recordings

Matt is a member of the NDS Labs technical advisory committee and will present NDS Labs as a platform for exploring data services -- enabling the separation of data and its representation, and how NDS Labs is functioning as an emerging platform for such separation.