Geospatial Data Accuracy and Standards: A Comprehensive Overview, Slides of Geochemistry

Download Geospatial Data Accuracy and Standards: A Comprehensive Overview and more Slides Geochemistry in PDF only on Docsity! 7/21/2013 Ron Briggs, UTDallas GISC 6381 GIS Fundamentals 1 Data Quality GiGo: garbage in, garbage out ‘Cos it’s in the computer, don’t mean it’s right It’s not the things you don’t know that matter, it’s the things you know that aren’t so. Will Rogers, Famous Okie GI specialist “But there are also unknown unknowns: the ones we don't know we don't know.” Donald Rumsfeld “Fast is fine, but accuracy is everything.” Wyatt Earp Docsity.com 7/21/2013 Ron Briggs, UTDallas GISC 6381 GIS Fundamentals 2 Horwood’s Short Laws on Data Dr. Edgar Horwood, founder of the Urban and Regional Information Systems Association (URISA) and Professor of Civil Engineering and Urban Planning at the University of Washington was an early pioneer of computer mapping in the early 1960s. • Good data are the data you already have. • Bad data drives out good. • The data you have for the present crisis was collected to relate to the previous one. • The respectability of existing data grows with elapsed time and distance from the source of the data. • Data can be moved from one office to another but cannot be created or destroyed. • If you have the right data, you have the wrong problem; and vice versa. • The important thing is not what you do but how you measure it. • In complex systems there is no relationship between the information gathered and the decision made. • The acquisition of knowledge from experience is an exception. • Knowledge grows at half the rate at which academic courses proliferate. Docsity.com 7/21/2013 Ron Briggs, UTDallas GISC 6381 GIS Fundamentals 5 Scale Examples Common Scales 1:200 (1”=16.8ft) 1:2,000 (1”=56 yards; 1cm=20m) 1:20,000 (5cm=1km) 1:24,000 (1”=2,000ft) 1:25,000 (1cm=.5km) 1:50,000 (2cm=1km) 1:62,500 (1.6cm=1km; 1”=.986mi) 1:63,360 (1”=1mile; 1cm=.634km) 1:100,000 (1”=1.58mi; 1cm=1km) 1:500,000 (1”=7.9mi; 1cm=5km) 1:1,000,000(1”=15.8mi; 1cm=10km) 1:7,500,000(1”=118mi); 1cm=750km) Large versus Small large: above 1:12,500 medium: 1:13,000 - 1:126,720 small: 1:130,000 - 1:1,000,000 very small: below 1:1,000,000 ( really, relative to what’s available for a given area; Maling 1989) Map sheet examples: 1:24,000: 7.5 minute USGS Quads (17 by 22 inches; 6 by 8 miles) 1:7,500,000 US wall map (26 by 16 inches) 1:20,000,000: US 8.5” X 11” Docsity.com 7/21/2013 Ron Briggs, UTDallas GISC 6381 GIS Fundamentals 6 Scale, Resolution & Accuracy in GIS Systems • On paper maps, scale is hard to change, thus it generally determines resolution and accuracy--and consistent decisions are made for these. • A GIS is scale independent since output can be produced at any scale, irrespective of the characteristics of the input data— at least in theory • in practice, an implicit range of scales or maximum scale for anticipated output should be chosen and used to determine: – what features to show • manholes only on large scale maps – how features will be represented • manhole a polygon at 1:50; cities a point at 1:1,000,000 – appropriate levels for accuracy and precision • Larger scale generally requires greater resolution • Larger scale necessitates a higher level of accuracy • GIS also helps with the the generalization problem implicit in paper maps – A road drawn with 0.5 mm wide line (the smallest for decent visibility) • At 1:24,000 implies the road is 12 meters (36 feet) wide • At 1:250,000 implies the road is 125 meters (375 feet) wide – At least in a GIS you can store the true road width, but be careful with plots! Docsity.com 7/21/2013 Ron Briggs, UTDallas GISC 6381 GIS Fundamentals 7 Precision or Resolution it’s not the same as scale or accuracy! Precision: the exactness of measurement or description • the “size” of the “smallest” feature which can be displayed, recognized, or described • Can apply to space, time (e.g. daily versus annual), or attribute (douglas fir v. conifer) • for raster data, it is the size of the pixel (resolution) – e.g. for NTGISC digital orthos is 1.6ft (half meter) • raster data can be resampled by combining adjacent cells; this decreases resolution but saves storage – eg 1.6 ft to 3.2 ft (1/4 storage); to 6.4 ft (1/16 storage) • resolution and scale – generally, increasing to larger scale allows features to be observed better and requires higher resolution – but, because of the human eye’s ability to recognize patterns, features in a lower resolution data set can sometimes be observed better by decreasing the scale (6.4 ft resolution shown at 1:400 rather than 1:200) • resolution and positional accuracy – you can see a feature (resolution), but it may not be in the right place (accuracy) – higher accuracy generally costs much more to obtain than higher resolution – accuracy cannot be greater (but may be much less) than resolution (e.g. if pixel size is one meter, then best accuracy possible is one meter) 1.6ft 3.2ft 3. 2f t Docsity.com 7/21/2013 Ron Briggs, UTDallas GISC 6381 GIS Fundamentals 10 Measurement of Positional Accuracy • usually measured by root mean square error: the square root of the average squared errors • Usually expressed as a probability that no more than P% of points will be further than S distance from their true location. • Loosely we say that the rmse tells us how far recorded points in the GIS are from their true location on the ground, on average. • More correctly, based on the normal distribution of errors, 68% of points will be rmse distance or less from their true location, 95% will be no more than twice this distance, providing the errors are random and not systematic (i.e. the mean of the errors is zero) – e.g. for NTGISC digital orthos RMSE is 3.2 feet (one meter) for USGS Digital Ortho Quads RMSE spec. is approx. 33 feet or 10 meters (but in reality much better) -- with GPS, height is 2 or 3 times less accurate in practice at high precision than horizontal (officially the spec is 1.5, but data collection errors affect vertical the most) e12 + e22 + e32 +...+ en2 n-1 rmse = where ei is the distance (horizontally or vertically )between the tue location of point i on the ground, and its location represented in the GIS. Docsity.com 7/21/2013 Ron Briggs, UTDallas GISC 6381 GIS Fundamentals 11 National Map Accuracy Standards: 1941/47 • established in 1941 by the US Bureau of the Budget (now OMB) for use with US Geological Survey maps (Maling, 1989, p. 146) • horizontal accuracy: not more than 10% of tested, ‘well defined’ points shall be more than the following distances from their true location: – 1:62,500: 1/50th of an inch (.02”) – 1:24,000: 1/40th of an inch (amended to 1/50=.02” in 1947) – 1:12,000: 1/30 of an inch (.033”) • Thus, on maps with a scale of 1:63,360 (1”=1 mile) 90% of points should be within 105.6 feet [(63360 X .02)/12)] of their true location. • on USGS quads with a scale of 1:24,000 (1”=2,000ft) 90% of points should be within 40 feet [(24,000 X .02)/12] of their true location. • on a map with a scale of 1:12,000 (1”=1,000ft), 90% of points should be within 33 feet (1,000 X .033), approx. 10 meters • gives rise to the loose, but often used, statement that the “NMAS is 10 meters” • Inadequate for the computer age – how many points? how select? – how determine their ‘true’ location – what about attribute completeness? • Unfortunately, the “new standard” doesn’t address all these issues either 1:20,000 1/50=.02” 1/30=.033” Smaller scale Larger scale Docsity.com 7/21/2013 Ron Briggs, UTDallas GISC 6381 GIS Fundamentals 12 National Standard for Spatial Data Accuracy (NSSDA) 1998 Geospatial Positioning Accuracy Standard (FGDC-STD-007) Part 3, National Standard for Spatial Data Accuracy FGDC-STD-007.3-1998 • “replacement” for National Map Accuracy Standard of 1941/47 • specifies a statistic and testing methodology for positional (horizontal and vertical) accuracy of maps and digital data • no single threshold metric to achieve (as with old Standard), but users encouraged to establish thresholds for specific applications • accuracy reported in ground units (not map units as in 1941 standard [1/30th inch]) • testing method compares data set point coordinate values with coordinate values from a higher accuracy source for readily visible or recoverable ground points • altho. uses points, principles apply to all geospatial data including point, vector and raster objects – other standards for data content will adopt NSSDA for particular spatial objects • copies of the standard available at: http://www.fgdc.gov • Accuracy Standard has 7 parts, of which parts 4-7 apply to specific data types Docsity.com 7/21/2013 Ron Briggs, UTDallas GISC 6381 GIS Fundamentals 15 Examples of Accuracy Docsity.com 7/21/2013 Ron Briggs, UTDallas GISC 6381 GIS Fundamentals 16 Lineage • identifies the original sources from which the data was derived • details the processing steps through which the data has gone to reach its current form • Both impact its accuracy • Both should be in the metadata, and are required by the Content Standard for Metadata (see below) • Michael Goodchild ( the guru of GIS) advocates: – Measurement-based GIS, in which how data collected and how measurements made are a part of the record (as in surveying) – Coordinate-based GIS, is the current approach, and it tracks none of this. Docsity.com 7/21/2013 Ron Briggs, UTDallas GISC 6381 GIS Fundamentals 17 Currency: Is my data “up-to-date”? • data is always relative to a specific point in time, which must be documented. – there are important applications for historical data (e.g. analyzing trends), so don’t necessarily trash old data • “current” data requires a specific plan for on-going maintenance – may be continuous, or at pre-defined points in time. – otherwise, data becomes outdated very quickly • currency is not really an independent quality dimension; it is simply a factor contributing to lack of accuracy regarding – consistency: some GIS features do not match those in the real world today – completeness: some real world features are missing from the GIS database Many organizations spend substantial amounts acquiring a data set without giving any thought to how it will be maintained. Docsity.com The Process for Setting de jure standards! Source: URISA News Issue 197, Sept/Oct. 2003 Docsity.com 7/21/2013 Ron Briggs, UTDallas GISC 6381 GIS Fundamentals 21 Adopting Standards: What you should do • Data quality achieved by adoption and use of standards: Do it! – Common ways of doing things essential for using & sharing data internally and externally • only federal agencies required to use FGDC standards, its optional for any others (e.g. state, local) – power of feds often results in adoption by everybody, although there are some noted failures (e.g.the OSI, GOSIP, & POSIX standards in computing in the 1980s failed and were withdrawn) • FGDC or ISO standards provide excellent starting point for local standards, and should be adopted unless there are compelling reasons otherwise • Standards for metadata (“documenting your data”) are the most important and should be first priority. – Content Standard for Digital Geospatial Metadata (version 2.0), FGDC-STD-001-1998 – ISO Document 19115 Geographic Information-Metadata (content) and 19139, Geographic Information—Metadata—Implementation Specification, (format for storing ISO 19115 metadata in XML format) – If not one of these standard for metadata, adopt some standard! Docsity.com 7/21/2013 Ron Briggs, UTDallas GISC 6381 GIS Fundamentals 22 Content Standards for Digital Geospatial Metadata What and Why? Metadata — describes the content, quality, format, source and other characteristics of data. Allows you and others to: – Locate data (find, discover) – Evaluate data (quality, restrictions, reputation) – Extract (order, download, pay) – Employ (apply, use) and automate this process. Docsity.com 7/21/2013 Ron Briggs, UTDallas GISC 6381 GIS Fundamentals 25 Texas Standards http://www.dir.state.tx.us/tgic/pubs/pubs.htm • Standards for digital spatial data (raster and vector) for State agencies in Texas were established in 1992 – Currently (2004), being reviewed by the Texas Geographic Information Council (TGIC) for possible update – Apply to map scales of 1:24,000 and smaller (e.g., 1:100,000; 1:250,000). – Cover variety of issues including data layers, datum, projections, accuracy, metadata, etc.. • Two major planning reports on GIS in state gov. in Texas are: – Digital Texas: 2002 Biennial Report on Geographic Information Systems Technology – Geographic Information Framework for Texas (1999) Docsity.com 7/21/2013 Ron Briggs, UTDallas GISC 6381 GIS Fundamentals 26 Importance of Standards  Great Baltimore Fire of 1904 - fire engines from different regions responded only to be found useless since they had different hose coupling sizes that did not fit Baltimore hydrants - fire burned over 30 hours, resulted in destruction of 1526 building covering 17 city blocks.  Fire 1923 - Fall River, MA saved when over 20 neighboring fire department responded to a town fire since they had standardized on hydrants and hose couplings sizes.  9/11: Response in NY and DC severely hampered by  incompatibilities between GIS data sets, and lack of data  Also, incompatibilities between communications systems  The most important standard?  Railroad track gauge - adopted by US, UK, Canada, and much of Europe.  South America still hampered by differing railroad gauges between countries. Docsity.com 7/21/2013 Ron Briggs, UTDallas GISC 6381 GIS Fundamentals 27 The Best Time to Adopt a Standard? Before! Now? Now? Docsity.com 7/21/2013 Ron Briggs, UTDallas GISC 6381 GIS Fundamentals 30 FGDC: Data Accuracy Standard Geospatial Positioning Accuracy Standard (FGDC-STD-007) Part 1, Reporting Methodology FGDC-STD-007.1-1998 Part 2, Geodetic Control Networks FGDC-STD-007.2-1998 Part 3, National Standard for Spatial Data Accuracy FGDC-STD-007.3- 1998 Part 4: Architecture, Engineering Construction, and Facilities Management (FGDC-STD-007.4-2002), Part 5: Standard for Hydrographic Surveys and Nautical Charts (Review) •An umbrella incorporating several accuracy standards. •Part 3 is the general standard. •It essentially updates the National Map Accuracy Standard of 1941/47 Docsity.com 7/21/2013 Ron Briggs, UTDallas GISC 6381 GIS Fundamentals 31 FGDC: Data Content Standards • Cadastral Data Content Standard FGDC- STD-003 • Classification of Wetlands and Deep Water Habitats FGDC-STD-004 • Vegetation Classification Standard FGDC- STD-005 • Soils Geographic Data Standard, FGDC- STD-006 • Content Standard for Digital Orthoimagery, (FGDC-STD-008-1999) • Content Standard for Remote Sensing Swath Data, (FGDC-STD-009-1999) • Utilities Data Content Standard, (FGDC-STD- 010-2000) • NSDI Framework Transportation Identification Standard, (Review) • Hydrographic Data Content Standard for Coastal and Inland Waterways, (Review) • Content Standard for Framework Land Elevation Data, (Review) • Facility ID Data Standard, (Review) • Address Content Standard (Review) • US National Grid (FGDC-STD-0011-2001) • Earth Cover Classification System, (draft) • Geologic Data Model, (Draft) • Governmental Unit Boundary Data Content Standard, (Draft) • Biological Nomenclature and Taxonomy Data Standard (draft) • National Hydrography Framework Geospatial Data Content Standard (proposal) • Environmental Hazards Geospatial Data Content Standard, (dropped) • NSDI Framework Data layers (under Review—see next slide) Docsity.com FGDC: Framework Data Standards • establish data content requirements for the seven layers of geospatial data that comprise the National Spatial Data Infrastructure (NSDI), the base layers needed for any geographic area • Goals are to – Facilitate and promote exchange of framework layers between producers, consumers, and vendors thru a common content and way of describing that content – Lower the cost of data for everyone • For each layer, specifies an integrated application schema in Unified Modeling Language (UML) including feature types, attribute types, attribute domain, feature relationships, spatial representation, data organization, and metadata • no standard specified for data format, but an appendix describes a possible implementation using the Geography Markup Language (GML) Version 3.0, developed through the Open GIS Consortium, Inc. (OGC). • geodetic control, • elevation, • Orthoimagery • Hydrography (water) • Transportation • Cadastral (landownership) • governmental unit boundaries Docsity.com