misc-gis

CMUG 09-16: Catonsville

2009-09-10T12:07:00.000-07:00

Apologies for the late post; I was temporarily blocked by blogspot.

We'll be having our fall meeting this coming Wednesday (9/16) at CCBC-Catonsville in Building Q's theater lounge.

We've got 3 great presentations lined up:

Jeff Cox, ESRI's Authorized Training Program in Anne Arundel County

Jaimer Alveraz, Where Are They All Coming From?: Mapping the Community College of Baltimore County’s Student Population

As always we'll have a technical demonstration; tentatively, it's planned to be on ESRI's new ArcGIS Explorer

The full agenda and driving directions can be found here.
See you there!

James & Sarah

ESRI UC: Thursday afternoon & Friday morning

2009-07-20T05:35:00.000-07:00

Geoprocessing: The Road Ahead

Thursday afternoon featured one of the sessions I had been most interested in attending- Geoprocessing- the Road Ahead. From what was featured in the Plenary and the Desktop Road Ahead, it's been clear that a lot of effort has been placed in enhancing the geoprocessing analysis tools and making them easier to use.

The presentation expanded on the themes already described in other sessions:
1) A new user experience
2) ModelBuilder enhancements
3) Python incorporation
4) New tools

As seen in the Plenary and anytime you looked at 9.4, a new Geoprocessing menu is present, pre-loaded with some of the most commonly used tools. This menu is customizable, so you can place your own favorites in the menu. The tools will be findable under search and has had the search terms expanded; searching for 'cookie-cutter' will bring up the Clip tool. Taking another nod from the server environment, the Results window (you use that, right?) has been moved to its own, hideable window. Instead of a command-line window, a full featured python window is available.

ModelBuilder has gotten a few enhancements which will enable for more complexity in the models. Iteration types have been expanded, with built in support of 'all files in folder' and 'all feature classes in workspace' type for each loops. The model-specific tools (Select Layer..., etc.) can be found in a special menu in ModelBuilder (Model Tools)- making it easier to find these utilities (and simultaneously remove them from the interactive geoprocessing experience). The user interface is also improved; an undo/redo button set (including model layout changes) has been added and tool tips for parameters now appear when hovering over a tool. Particularly useful to me is the 'pop-up connection' feature, which will let you specify which parameter an input should be directed to when making the connection between them.

As has been announced, Python is the scripting library for ArcGIS. A lot of work has been done to make the 9.4 ArcGIS Python library more 'pythonic'- it's now going to be a module (arcpy), so we won't need to instantiate a geoprocessing object. There are three main packages: mapping, geoprocessing, algebra (as in map algebra). ArcGIS will also be able to expose external libraries (such as the r statistical package) as tools- so you can place your custom code as simple menu items. As part of making the arcpy module, intellisense support and pyDoc documentation has also been enhanced.

Some features of the mapping library were highlighted. Essentially, all properties of an MXD document will be exposed, and custom maps can be made via scripting. We saw PDF support- I'd like to know what other formats will be supported- all of the export options?

The new analysis tools were featured. Right now, there are 90 of them; the development team thinks it might be above 100 by the time 9.4 ships. A big new grouping of tools are 'Editing tools'- think some of the editing features (e.g., trim, extend) being applied en masse on an entire feature class or selection set.

A lot of tweaking has been done with the Spatial Analysis tools to improve performance and usability, and classic map algebra has been reintroduced. New is the concept of a 'fuzzy overlay'- instead of producing a determinate output, a probabilistic result is generated; this support set theory's AND, OR, and GAMMA (links) functions. Image analysis has also gotten easier with 9.4; in particular, it's easier to create a classification training set and analysis can be done on just the data in the view, making it easy to prototype the analysis.

Like Spatial Analyst, Geostatistical Analyst has also had the UI and defaults improved to make it more ready to use and provide meaningful results. Also, ecological sampling tools have been provided- including a tool to determine additional sampling locations based on kriging standard areas (this was developed by the EPA to improve analysis capabilities over the classic random points).

The 3-D features have been improved; aside from the new tools covered in the Plenary, all the 3-D analysis tools support terrains as well as TINs. Also, some tools to help improve native LiDAR processing have been included, such as being able to identify outliers and blunders in point data and remove points under polygons (water masks).

A lot of the improved functionality in Network Analyst were featured in the Geodatabase Road Ahead session. 3-D routing will also be available. Service Area and Closest Facility processing will be available via REST; this will make a lot of Flex applications more fun!

ArcGIS Explorer: Beyound the Basics

My office has played with ArcGIS Explorer (AGX) off and on over the past couple of years. With the AGX 900 release, it looks like it's time again. AGX is ESRI's desktop product that they're targeting at 'Geographic Information Users'- the people who need to browse GIS data but necessarily do complex analysis. The new features of AGX (Geopresentations/2D-3D/etc) were briefly mentioned (those would be the basics we're going beyond); this session covered how to customize and extend the application to make it fit your particular organization's needs. Some data setup tips (that are applicable in ArcGIS Desktop and KML export as well) were covered. First, in order to keep the download small, the AGX core software is missing a few things, like fonts and full projection support. These can be separately downloaded as expansion packs. Second is the reminder that you should spend some time on layer configuration- 3-D layers need to be authored in ArcGlobe, HTML pop-ups should be enabled and set how you want users to see their data.

All I can really say is that everything is extraordinary simple to set up and implement. You can build new base maps, implement custom splash screens, and customize the toolbars without any programming experience- the first can be done within Explorer, the latter two through an Application configuration manager that ships with it. If you want to develop your own tools, you can create 'add-ins' in .NET- the SDK has a wizard to start the process and take care of the more annoying programming overhead, so you can get right to writing the add-in's functionality. Add-ins will support Buttons, Dockable Windows, Galleries, and Extensions; they're bundled in .eaz files for easy transport and saved in a 'well-known' folder location (i.e., everybody has the same folder structure, so we know were to look).

Server Optimization

Optimization by it's nature is a 'tips and tricks' type of session, and it's better to look at the slide deck than rely on anybody's summary. One of the interesting things I took out of it is that we really are running a small setup (web/SOM/SOC box pulling in data from a separate SDE server). A lot of the cases shown involved optimizing multiple and even 8-server+ configurations.

Closing session

Ah, the end. If you've been at a closing session at and ESRI conference before (UC or Dev Summit), this should be familiar. The winners from the Map Gallery were announced (Walks in Portland, Irish names, San Luis Obispo all pretty prominent). Jack re-iterated the concept of GeoDesign. Through the questions raised at the conference and at the Q & A, a big theme emerged- Community, providing ways to enable GIS users of all levels to connect continuously and share what they do. It sounds like ESRI will be spending a lot of time in that in the future.

A new part of the Q&A process involved voted-on video questions. The top vote getter was what were ESRI's plans after 9.4. While there is still nearly a year to go before 9.4 is released, the big evolution that is being considered is moving to marketing ESRI (the company) as a platform or product. I'm honestly surprised they haven't already; at least in my area (DC), they do a fair amount of professional services, training, and other consulting.

Some improvements to the conference format were suggested. For those of you who found the rooms to warm, you'll be disappointed that someone thanked the staff for not freezing them this year (I concur, even if I found the rooms a little cooler than I like- I'm just used to DC's 90 degrees - 90% humidity summer climate). A way to pre-screen, recommend, or vote on paper presentations was also suggested; the paper sessions can sometimes be of variable quality. Likewise, better searching for the sessions would be useful; at the end of my paper presentation (a somewhat technical one on Python), I had someone approach me and ask why I wasn't in the program. It turns out he was looking through the Technical Workshops section- the ESRI provided 'our software and how to use it' sessions, instead of the Paper Presentations.

A couple of questions about ESRI as a company and how it positions itself in (and thus structures) the GIS industry. Jack's response on being a private company is classic- "running a private company is horrible but it's better than the other options". Jack sees the company as providing the services to to the GIS professionals (and some prosumers), who will then develop the maps and tools used by the consumers (rather than targeting the consumers of GIS data directly). ESRI is very excited to support NGS's Geomentoring program to place GIS into primary and secondary education, and encouraged us to volunteer.

This was a busy conference, and while I learned a lot, I also came away with some questions that I hope to explore a bit more thoroughly in the future.

Wednesday/Thursday morning at the conference

2009-07-16T11:31:00.000-07:00

I gave my presentation first thing in the morning (see the post below).

I attended the Geodatabase Road Ahead session- AnyGeo has a good, quick summary. While there's a lot going on under the hood, it seems like the SDE featureset is fairly stable. The expanded support in the network dataset for multiple shape types of blockages, increased cost areas, and time-dependent costs will be very useful; the next logical step is to have an implementation of multiple cost types (i.e., roads and tolls). The new 'child to parent' replication method will be extremely useful in simplifying federated database workflows (each county can directly push their work to a complete state dataset if they can handle most of the ETL actions through a schema cross-mapping). At the end of the session, I asked about 9.x version interactions- the parent will need to be 9.4, but the child won't (the state can establish the shiny new DB and the counties can transition as their budget allows).

In the afternoon, I took advantage of a couple of the non-session activities available. A couple of my co-workers are developing a new print map template at work, and I went to the Map Critique area with the current draft of the base map layers (the bread of the 'map sandwich'). I got a lot of feedback and had a good conversation with ESRI cartographic staff.

I also dropped by the Hands-On Training Lab and took a free 45-minute course (it was a topic I already had some user knowledge but not a thorough background in). The format was fantastic- a 30 minute video presentation (which I admittedly used the 'play faster' option to get through in 20) and then a set of exercises to give you the physical interaction. I hope they develop more of these in the future and I should look to using the format for my own in-house training.

I also got to try out the 9.4 Beta software- it looks pretty good. In particular, I was interested in the 'basemap group layer'- you can have multiple basemap layers in the map document (say the background in one and borders/annotation in another); they'll both draw continuously as you pan (the base map, which was fairly data heavy, took a second to render for areas where drawing hadn't occurred before). Also, when a layer is placed in a basemap group, it won't interact with the identify button- I actually like that behavior (too often, the top-most layer is relatively non-important boundary). I like the interface- it really does allow for a lot customization and hides everything you don't want.

This morning I caught the "Enterprise DB Tips & Tricks" session. I'm not the DB guy at work, but I work with him extensively and want to at least have a good understanding of the issues he has with maintaining the SDE (and occasionally give him a semi-informed suggestion). While I haven't personally dealt with the issues presented, the information did help tremendously- it put into overall system context the operations that I see at work.

My Talk: Extending Python Scripts

2009-07-16T11:25:00.000-07:00

So, I gave a talk yesterday "Extending Python Scripts" (it's 5MB). I'm a big fan of using Python for geoprocessing (I'm naturally psyched about the expansion of Python capabilities coming up in 9.4).

The ability to make maps in 9.4 through Python highlights one of its current shortcomings- an analysis process must provide output that is readable to user. If you've designed the analysis, great; however, if you're deploying the process for someone else, the outputs need to be readily usable in multiple situations.

A similar situation occurs in ArcMap. One of the ways ESRI solves this is by including charting functionality and bundling Crystal Reports, an external helper tool.

In the same way, we can incorporate external libraries into our Python scripts to let us process the results of the analysis into a more usable form. This particularly useful when designing scripts for use in a server context; you are much less certain of what the user has available for tools to re-process the analysis.

In my talk, I covered 5 libraries that I've used in geoprocessing scripts:
-ReportLab, a PDF library
-xlwt, an Excel writing utility
-PIL, an image processing utility
-urllib, built-in library to grab stuff via the web
-smtplib, built-in library to send e-mails

Of these, I feel that the urllib is the most powerful. With it, you can grab data from ArcGIS Server REST sources as well as other websites. This allows you to do create mash-ups within the geoprocessing applications!

The other libraries all in some way deal with providing output. Excel and PDFs are more user friendly than shapefiles or dBase tables and more widely accepted in consumer client situations. smtplib provides the capability to e-mail the results, rather than have them displayed at the time of analysis; we've used this at public displays, where people can e-mail a report of their house to themselves while talking with our planners.

My example scripts are available at the Model and Script Tool Gallery.

Re-blog: Other session reports I'm interested in

2009-07-15T10:09:00.001-07:00

Road Ahead- Desktop/Server/Mobile/: http://geo.geek.nz/events/arcgis-desktop-9-4-arcgis-server-9-4-and-arcgis-mobile-9-4-the-road-ahead-at-the-esr-international-user-conference-2009/ (good quick overview)

Server Road Ahead: http://ambergis.wordpress.com/2009/07/15/esri-uc-arcgis-server-9-4/ (covers pretty much all I would cover)

Flex SIG: http://veryspatial.com/2009/07/15/esri-uc-day-2-roundup-couple-of-sessions/. It *was* a packed session. I wasn't thrilled with the 'programmers can't design' attitude- it's not my strong suit, but a lot of geographers do have at least some cartography experience.

Desktop: The Road Ahead

2009-07-15T09:50:00.001-07:00

A lot of new technology gets demonstrated at the plenary very briefly. To compensate and complement this, there are a series of 'Road Ahead' sessions. Given the wide arrays of technologies shown, it's not surprising that there are several separate Road Ahead talks. I'm planning to catch the Desktop, Server, Geoprocessing and Geodatabase sessions. I saw the first two today (Unfortunately, the first Geoprocessing Road Ahead is the same time as my paper; it's a good thing there are 2 talks for each session).

The Desktop Road Ahead was the first session of the morning. The continuing theme is that 9.4 is an 'evolution' of the ArcGIS platform, not a major change. At the plenary, it was noted that there had been some debate as to whether to call this release 9.4 or 10 (indicating a significant change). If ArcGIS was still overwhelming a desktop product, the '10' argument would probably have won out; while ArcMap is still recognizably ArcMap, some serious changes have happened in the UI & under the hood.

The first thing they covered was the big question unanswered at the plenary- the timeline for 9.4. They're expecting a large, open Beta program this fall. I asked about eligibility at the Beta Island on the floor; it sounds like everyone who has an attendee at UC is going to be automatically enrolled in the Beta (I don't know if they have enough "bug buster" shirts for everyone, though). A pre-release candidate should be due early 2010, and 9.4 release in spring, with fully internationalized versions in 5-6 major languages (I missed the list, but the slide was showing Kanji) a few weeks later.

A key concept in the design of ArcMap was to preserve the current GIS workflows; unlike Office 2007, which fundamentally changed how things are done within that program. The menus and toolbars have been 'tidied up'; The Add Data entry on the file menu has grown to accommodate geocoding and x,y tables (though the button on the toolbar didn't seem to reflect this), the 'Tools' menu has been reduced to just the customization options and logically renamed and a Geoprocessing menu has been added. Geoprocessing tools are now accessible as buttons throughout the the UI, whether on a toolbar or in a menu item; this includes custom tools like models and script tools.

The entire look and feel should be very familiar if you've used Visual Studio, especially VS 2008. This is an interesting contrast to the use of the 'ribbon' interface in ArcGIS Explorer- it seems to underscore the notion of the expertise differential between the two program's users (everyone uses Office/AGX while not everyone uses VS/ArcMap). Anything that generates a persistent window can dock it's window inside the containers by dragging the window to the container and then selecting the location with positioning arrows. Panes can also be set to auto-hide, so that the map area is normally maximized. This includes a 'Catalog' window for browsing for data By default, ESRI wants us to work in an environment where the maps, data, other documents and perhaps even tools or other supporting GIS elements are stored together in one directory for the project- there are some browsing shortcuts that play off this notion. The catalog window in ArcMap doesn't create a separate lock on a gdb when accessing it, eliminating a common frustration of users- having ArcCatalog and ArcMap (separate applications) work on the same data is impossible. Also included is a 'Search' window that can be used to search for data (metadata or tags), maps and tools (styles/symbols also have a search interface, so you don't need to add them & then hunt for the right symbol). Another welcome addition is background geoprocessing, so you can start tasks and get back to your work (I asked: only one background GP process at a time).

Drawing performance has been significantly improved- I'm going to take a guess that they've moved over to the faster graphic engine that's responsible for the optimized map services in 9.3.1. That would explain the support for Maplex and Representations in the map services. A major new feature is the concept of a 'basemap' group layer- you place your background layers into a basemap group layer and it automatically caches the features and accelerates their drawing- the base map provides a continuous pan experience. I haven't gotten the full details on it, but I followed up with the staff member presenting it later- the basemap layers are pressed into images and feature querying is no longer possible, so I'm guessing that ArcMap is making a much more limited request from the geodb than a normal redraw takes (just enough to to process symbology, maybe- shape & categorizing data?). The display engine still needs some work, though- at several times in the presentation, a refresh had to be manually invoked.

A lot of tweaks have gone into working with features. Interactive selection has been beefed up, so that you can select by point, rectangle, or arbitrary area. The Selection tab is being replaced by 'Layers', which will make management of selectability and selection sets more obvious and better integrated with how the data is classified- in fact, I think the Layers tab will be the default tab for many people, rather than the TOC display. The editing environment now makes use of templates, again based on a classification set, which can control which fields are shown during editing and pre-fill values. These editing templates are applicable in both web and desktop contexts. Snapping is now a separate toolbar and more easily provides control and feedback on what features you're snapping to (and seems a lot quicker). Snapping is also available for all tools, so georeferencing scanned plats, for example, has just gotten a lot easier. The editing tools work in ArcScene in a 3-D environment and the editing tools are now unified for features, annotation and dimensioning.

One thing noted is that Mapping & Automation don't necessarily sound like they go together. If you've had to do a series of ~300 park maps you'd know otherwise. Thus the expansion of Python scripting into mapping and the major new feature in the Layout, data-driven pages. From talks after the presentation, it sounds like the data-driven pages is the tip of the iceberg that can be done with python. Essentially, data driven pages creates a series of maps based on a layer. Any layer- a uniform grid, points, arbitrary polygons. To get a uniform grid, there is now a "Create mapping grid" tool which creates a grid based on the extent/presence of a feature class. Just like any other multi-page output function, there are data-driven fields- the default ones are a little limited (relating only to environment, position within series, and the feature controlling the data), but can be expanded using python access to layout graphics. Python can also access properties of the map and layer documents; the big example is a bulk repointing of data locations (Find/replace *.shp -> SDE :) ). More will be in the Geoprocessing Road Ahead session, but it is absolutely clear in my mind that Python is the new AML.

They wrapped the talk with showing the RoadMap for developers/administrators. License management will be more flexible- license checkout will be supported, so you can log in first thing in the morning, check out a license, and hog it even if you don't have ArcMap. Win2K is not supported, Win7 will be. If you're doing .Net development, you'll need VS 2008 or 2010; VB 6 is dropped and VBA will be a separate install to start the withdrawal process.

From a development standpoint, this should be an easier transition- both 9.3.1 and 9.4 can be installed concurrently, so you can still use the 9.3.1 libraries. A *lot* of customization can be done through simply modifying the layer properties and ArcMap document, so some of the basic VBA tasks have disappeared. Python/Geoprocessing will be the environment of choice for analysis. For more complicated GUI modifications, "add-ons", written in .NET/Java but avoiding a lot of the registration overhead, can be developed and placed at a set location. I'm assuming ArcMap will load them, wrap them in a sandbox that handles application registration, and use them as if they were normal custom code. Unfortunately, I haven't seen an example of those yet.

All in all, I'm excited- it's a shame it's nearly a year out.

ESRI UC Plenary Thoughts

2009-07-14T05:47:00.000-07:00

The plenary can probably be best broken into 3 themes: what we're doing with GIS, what has been released, and what's coming in the future.

The award winners, highlighted by CenterPoint Energy and the State of Maryland, are clearly showing what we're doing. Both of these organizations are working with the existing tools, maybe not to the limits of what the technology can do, but to the fullest extent that they are able to. A lot of what their providing is online in one form or another; it's clear that while the desktop clients are the main feature-rich interface, ESRI fully encourages online ('cloud') access for consumer use and general sharing.

A lot of time was spent showing what 9.3.1 has provided. It's a recently released product; a lot of groups are naturally slow to transition to it. It provides a lot of benefits in speeding up web mapping; it's also being complemented by the expanded data serving capabilities of ArcGIS Online (and the searching abilities of the GeoPortal extension to server, which I'm surprised *didn't* make the presentation, unless I somehow missed it). So that message is clear- for now, get up to 9.3.1.

By the same token, the ArcGIS Explorer portion was a major presentation- the 900 release is presumed to be in a 'few weeks' and the Explorer client is being placed as the desktop consumer client when an application running on a local computer is needed. I haven't had a chance to work with it, but it would be interesting to see what advantages it can provide in a small-area, local jurisdiction context over web mapping or ArcGIS desktop (the demos exhibited where of a global nature).

The 9.4 presentation was the natural 'wow' factor. ESRI really is spending a lot of resources into both the Desktop and Server products; the Desktop client in particular is getting a much-neede overhaul (I'd argue that overhaul came for Server at 9.3 with the REST/JavaScript/Flex/Silverlight setup). I didn't attend Developer's Summit, but I'm given to understand that a lot of the features shown had been announced then; what we were seeing was in some ways a refinement from this spring. Still, they are promising a lot. The entire experience of finding, getting, viewing and using data will be much improved- one app handles everything (and the UI is designed to let you concentrate on the particular action at hand). You can also see the influence of the server development coming into the desktop client, as the base map/operational layers separation is implemented to improve draw times and the geoprocessing tasks go asynchronous from ArcMap (I'd hate to be an analyst who mistakenly tries to have two transformative tasks simultaneously run on the same data, like two calculate fields; I wonder how the locking mechanism is going to work). Another server/web paradigm coming into desktop is a gigantic emphasis on searching- searching for data, tools, etc. This should make it easier for the technical staff to support users as most 'where is my data' questions can start to get resolved into a search.

Editing is also getting improved through refinements and focusing. We can intelligently construct frameworks around editing to ensure that we put in the stuff that needs manual input and let the computer handle the rest; both on the Desktop and in the many Server configurations. Python is fully inheriting the AML role as a scripting language, gaining cartography tools and enhanced analysis capabilities. This probably means tweaking the existing work you do, but generally for the better- it will take some time to accommodate (it's not surprising that you can have 9.3.1 and 9.4 installed on the same machine- the switchover time will probably at least a couple of weeks for most people as they get used to 9.4).

Most of the time on server GIS focused on the Mobile environment and the geodatabase. As I said above, 9.3 placed web mapping on a new, solid foundation. The mobile clients look to mature at 9.4 (especially the Windows laptop/tablet client). The creation of an open API for the geodatabase is a welcome sight for anyone storing data in an ESRI format and needing to get it to work elsewhere. The 'vector/raster' divide in GIS/RS package capabilities is continuing to narrow; now that the functionality is mostly in place, I wonder how ESRI plans to market themselves in the RS world.

The final new area was the support for increasingly complex forms of data. 3-D features will be much easier to work with and do analysis on (the distance-dependent rendering quality will make it much easier to to deal with complex scenes), and the instant importing/locating of multi-patches (such as imported Sketchup models) will be a great help for anybody who deals with the architectural realm- Sketchup doesn't make it easy to deal with a coordinate system, so having an easy way to place those models is nice. I'm glad the time analysis features are getting upgraded. An area of growth in our data is making features time-aware, as new laws, alter what a particular piece of data means. For example, zoning text amendments, while not changing the actual zone, can alter what can be done significantly; being able to see the change in the difference of development potential over time based on legislative history would be invaluable. Having a time-based lookup table is the easiest way to store the criteria for this analysis. I was also personally surprised by the georeferenced video layers- that could be a very cool add-on in GIS presentations, if not a really interesting way to bring real-time into GIS (think about using this with the CCTV network in Britain- rather than monitoring a ton of screens, you implement a mosaic of the fields of view).

Certainly, the presentation left as many questions as answers. The big ones in my mind is, of course, the timetable for 9.4. Many of the features they demo'd looked fairly mature, but there were also features mentioned but not really shown- was it just too much for one presentation or are they not ready for prime time? Should we be expecting SP2 or 9.3.2 with some of the more mature features?

Interweb training/tech support

2009-07-08T14:35:00.001-07:00

As part of my job I give about two in-house training seminars a month. Thos past month, I've used an online meeting service to host 'virtual' sessions. So far, this has worked pretty well- a few courageous coworkers signed up for it and I've received positive feedback.

Because GIS is so complex visually, one of the thongs I was concerned about was the quality of the presentation. In all honesty, the screen in the presentation is better than the projected image at the classroom- it's not going to a 2-year old, getting slightly burnt out portable projector not quite placed properly for display.

What I've liked:
• Easily adds seats without additional training hours- the room I do most of my training in only holds 8 people at a time and is away from our primary office building (working for a parks agency leads to a lot of off-sites). This provides a painless way to add class capacity and provide access to staff who can't make it to the training location.

• Screen control. The normal setup I've used is to have a set of machines for students to use as we go through the exercise and I project my screen. While this gives interaction time, the class slows down as I take numerous breaks to make sure everyoneis on the same page. The online people can follow along if the want independently, but most of them look at my screen; I can structure breaks for user interaction into the class.

• Nearly foolproof recording. The service Ive been using records the audio and video with a couple of clicks. I like to record my classes so that the students can refrence them and so I (or my bosses) can review and find ways to improve.

• Insta-meeting for tech support. Half the time on a phone call, I'm having to imagine the user's screen to describe what to do. We use VNC in the office, but not all the machines have it and getting the user to find their IP or machine name can be difficult. I can send them an e-mail *much* more quickly, and then flip the presentation to their screen.

What needs improvement:
• Log-in process. Maybe it's a good thing for the users to be apprehensive to install the client software- a sign that training has paid off. Still, I consistently hear users balk at "which e-mail do I use" to sign into the session. Ideally, a client could be designed to work off of directory if it detects a local network and use the current log-in credentials.

• Class structure. The online format requires some tweaking to build in user interaction, as opposed to allowing it to happen by following instructions.

• Equipment. This is a bit obvious, but to effectively host physical and online sessions simultaneously, make sure the room can function as a conference room. I had to scramble for a pair of speakers. Also, I was surprised at how many of my coworkers don't have computer headsets (yes, I'm a relatively young techie).

Grad School- worth it?

2009-07-01T18:16:00.001-07:00

Before I started my *real* job, I got a MA (UMD) & did some PhD coursework (UMN), as well as work in various university teaching/research/admin positions. I also was privileged enough to teach undergraduate courses this past year at UMD (programming & cartography), primarily because the full-time faculty were dedicated to the MPS in GIS program. I value the university's role in providing a base of GIS knowledge and skills. Still, the NY Times has a nice discussion on the value of grad school, and I think it's appropriate to view the MS/GIS programs in this light.

My personal thoughts echo the respondents. I'm in the DC/Baltimore area. They're local programs at UMD (~40-50 students/yr), UMBC (a guess at ~40/yr) and Salisbury University (a guess at ~20-30/yr), plus the GIS-oriented graduates of GMU, Towson, Penn State, just to name a few places with a local impact. I feel that the DC market is probably *the* biggest for GIS in the US (thank you, feds + sophisticated locals), but can we really readily absorb the production of bachelor's (conservatively, let's say 200) *and* 80+ MAs specializing in GIS (not to mention the people with GIS skills but not an explicit degree in it). If we can absorb that many MS students, is the payoff differential between a BS + 2-3 yrs experience and a MS anywhere near 7K/yr yet (that just might make the MA cost-efficient over 4 years, if you get funding)? I don't think so- from talking around, I've heard the usual horror stories of students being hired post-MA to do basic technician-level tasks; also, it takes some planning on an organization's part to integrate the combination of advanced skills and shallow experience that a former graduate student probably has.

The above being said, I think that a professional MA (or graduate certificate) has a major role in GIS- there simply isn't enough time or courses in an undergraduate curriculum to provide the breadth or depth of skills necessary to adequately grok certain topics (programming from a non-CS background, for example). Some of these topics are better learned either in a class or class/work setting (programming being very much the latter).

I just don't want potential students to go into debt and find that it takes more than 5 years to recoup a $50,000 MS (in-state full-time and bare-bones living)- it's hard on the beginning side of a degree to fairly estimate the expected payoff.

Wake Up

2009-07-01T18:13:00.000-07:00

Okay, it's time to try to reactivate this. I'm presenting @ UC (http://events.esri.com/uc/2009/infoWeb/OnlineAgenda/?fa=ofg_details_form&ScheduleID=227), after all :)

Stuck with a view of the Pacific

2008-03-21T11:25:00.000-07:00

Not yet home- I've been bumped twice (Easter, snow in O'Hare), so I'm spending the afternoon in SF- the tragedy of it all :) .

Anyway, my final dev summit opinions

One big benefit of the Dev Summit is that the session have all been recorded with screenshots; by all means, listen to any session you have the slightest interest in- it will be worth your while.

Building a 3-D City

9.3 is supporting 3-D in a big way. An "Import 3-D Model" tool was shown, with both Sketchup & COLLADA types, among others listed. The geometries supported by multi-patch (rings and triangle networks) were described, and enough relevant examples were presented to show when you'd want a ring (flat surface) or one of the two triangle sets (accommodate a continuous surface in 3-D). There's programmatic access to multi-patch, so a quick color/texture scheme can be easily applied.
Aside from that, ESRI is focusing on storing and managing 3-D models- not necessarily bulk creation- that's partner-space (though tools for individual buildings will be deployed, though it may be more sensible to use Sketchup or another 3-D design tool).
The Question & Answer was interesting. Among other things, it confirmed the role of ESRI's business partners in bulk (programmatic) 3-D creation, as well as some idea of what they're thinking about for 9.4- perhaps an architectural style symbol library and other ease of using existing 3-D data. ArcScene & ArcGlobe will continue in the foreseeable future.

Some tips that should have been obvious from the star (i.e., obvious to me but not):

Use Scale dependencies! The presenters recommended at most 5-6 different layers for use in a 3-D environment, with more complex models (kml, sketchup, multi-patch) being limited to the closest features. Use extrusion for slightly farther away, and footprints for even farther.

Use multiple footprints for a building to extrude properly. Most skyscrapers (or even mid-rise blocks) have differing elevations. Instead of making one solid building footprint, create a series of rings that complete the footprint and specify elevation information for each ring (separate the footprint component ID from the building ID).

Designing Map Caches

This was a session that I had planned to listen to after the conference, but decided to sit in. Aside from the ArcGIS Server guys, they also had a representative from ArcWeb Services (i.e., the guys who have done the most caches to date).
A lot of good take-away tips on building caches, including what image type to use (JPG for backgrounds, PNG8 for vector overlays) and what datasources to use (use a local GDB for tile creation; we can always repoint the data to an SDE afterwards).
The 9.3 interface for cache creation & maintenance looks pretty good; it'll be a lot more flexible (we're not hit too badly by it- our base background is generated in 8-20 hours, depending on image format).
This session also drove home to me the ESRI philosophy on map services: cache EVERYTHING that doesn't need to be more current than the time in which you can recache it (up to the hour or day can be cached, depending on your hardware). I'll bet this is going to result in a number of setups where a second(ary) server will be brought online to do constant caching, with maybe some excess capacity for hosting additional web serving SOC clients.

Effective Geodatabase Programming

Another session where the slide deck will become a great reference. I don't do too much programming against our database; however, the information at the session was pitched just low enough that I understood the concepts & reasons for their recommendations. I'll at least be able to be conversant with people I work with who do this stuff to convey the issues raised. One of the nice things about the Dev Summit is there was ample time in the schedule to take a session or two in topics that weren't my area of expertise- I honestly like cross-training in other areas.

Closing session

Basically, a good forum for feedback. A lot of hand-voting for what worked and what didn't (the intro sessions seemed to have helped, sit-down vs. boxed lunches was a draw). Some future directions were talked about; at this point, ESRI has nearly backed themselves into a corner to release a functional SDK for the File Geodatabase. Anything less then full access to it (without a ESRI DLL to be deployed in addition) was met with tepid response. Two other takeaways from the Q & A:

VB6 is moribund (if not dead), get over it

IMS is moribund (if not dead), get over it

Dev Summit Wednesday Sessions

2008-03-19T20:41:00.000-07:00

Architecting ArcGIS Server Solutions for Performance and Scalability

A series of tidbits, a few of which were useful (depended on your experience with serving map services). I wish they would have spent more time on computer/network architecture for hosting web services & apps- for me, the computers are the relatively scarce resource (especially when licensing per computer comes in), so we need to be able to figure out the best use of our computers.

Flex API Preview

Due to demand, this got moved over to a conference room, as opposed to a demo theater. I stayed a little while, but got enough of a gist: it looks to be pretty much a Flex wrapper for the REST API (I'm willing to be corrected if they showed stuff beyond the standard REST capabilities; I was in the session for all of 10 minutes because it was during a sit-down lunch.

This session re-enforced the big takeaway from the conference- Clients are whatever platform you want them to be with ArcGIS Server 9.3. By now, I'm sure there's some interface for ALGOL that can take advantage of the REST API (there better be, since it's http-based). The challenge for a development shop is to identify the 2-4 platforms (depending on the size of the shop) that they'll use to develop and actively keep the others out-of-bounds; otherwise, you can't adequately develop depth of programming to do good apps.

Building .NET Applications Using the ArcGIS Server Web ADF and ASP.NET AJAX

I was really excited to go to this session (I haven;t been to previous incarnations). Picking up from Dave Bouwman's comments, it really felt like REST let the air out of the room on this. The .NET Web ADF still allows pretty good access out of the box to heftier web services, but it's just about obvious that a lot of the energy is going to be over in the Javascript side of the camp. Still, the improvements in 9.3, which uses the Microsoft AJAX framework are pretty significant and much appreciated.

Building and Extending Tasks for ArcGIS Server .NET Web Applications

This session left me feeling more positive about the .Net Platform. I haven't developed any tasks yet, so the walkthroughs provided were extremely helpful. It also showed how the .Net ADF can continue- it provides a nice near-WYSIWIG interface for organizations with no programmers to continue to create & support (with outside code help) web mapping applications. Also, at 9.3, creating a task on a one-off basis becomes a lot easier by creating a web control & loading it as a tool, which will lesson the onus of developing a tool & Visual Studio configuration & web configuration interfaces.

Other

I had a good talk with ESRI Staff on geocoding; since I work for a local government, we're interested in maintaining our own, very particular address locators and had some problems tweaking them. I also got a good amount of time with the ArcPad team- I already love ArcPad as a mobile client (our people can be out in the field with no telecom access easily available), and it's only going to get better with 7.2.

Dev Summit Wednesday Keynote

2008-03-19T20:25:00.000-07:00

Apparently a Cooper nowadays designs the UI to programs, not barrels. Alan Cooper's talk was informative, especially to people who are (or want to) manage geeks. Talking points:

Making money moving atoms vs. making money moving bits- a little bit tired at this point to people who are used to computers; the key to remember with this point is that there are still many people who don't understand the difference.

Things like finance are now commodities/utilitarian in nature- having it (or having good implementations) don't help; not having hurts. Rejoinder: if that's the case, why do so many public companies still have bad financial practices?

Software isn't a commodity; it's a craft. My response: It isn't, yet. One of the big things Dev Summit has shown is that the bar for making really good online mapping applications with custom data is going to be lowered, quick. With other projects allowing for easy creation of mini-apps (Yahoo Pipes) and the mapping projects only being a part of software development), the bottom of the programming market is going to drop even faster as every high school sophomore is going to make their own app 'cause they can

The distinction between interface engineers, design engineers and production engineers is important. Controlling the UI (both software & human side), sketching out the functionality and finally making the code so it doesn't break are three separate tasks; I certainly can't do them equally well.

Finally: an Eisenhower quote: "In preparing for battle I have always found that plans are useless, but planning is indispensable."

A good keynote speaker, but probably not what every attendee was looking for (aside from reassurance that they're a special class of worker, for now).

Developing Geoprocessing on Server

2008-03-18T21:52:00.000-07:00

Or, how to do geoprocessing & make it work.

Despite being good with creating analysis processes, I'm still not completely comfortable with the Model Builder environment- it's not a complete scripting solution, and getting a model to run in a server environment has generally had problems. This session (admittedly partially given before over the web) was pretty much a best practices for developing to server (or developing a model in general) with an eye to what additional functionality is coming along in 9.3 (like layer transparency, to begin with). The slide deck (hopefully up soon) will definitely be a great reference.

Mentioned throughout the presentations today is the fact that pretty much any client, be it ArcMap, ArcExplorer, a Web ADF site, or even a mashup or python (or ruby, php, and pretty much any other language) program will be able to utilize a geoprocessing task- that really puts the onus on getting the task designed correctly in the first place.

REST & Mashups

2008-03-18T21:19:00.000-07:00

REST

It was packed, and naturally people more vocal than me have already posted (the first link, James Fee's, is especially good at explaining the technology).

Some notes:

The structure to access layers is well-defined, but a little human-unreadable (admittedly, it's a programming interface). It'd be nice to address a layer by an alias (and possibly not have to worry about a reordering of the map document breaking everyone's programs)

One of the things I'm interested in is a possible replacement for the IMS Metadata Service. I haven't taken a look at the metadata delivered, but I'll bet any replacement simply leverages the information published through the REST service.

The 9.3 release will be read-only, with writing probably(?) in 9.4

True curves (i.e. arcs) are densified (changed to line segments) when served out. While I wasn't completely clear on it, it sounded like that was standard across the ADF (or I'm being wishful; it currently causes a small failure).

Mashups

Another good session. Dave Bouwman has another good review up. I'd say it's even simpler than what Dave makes it out to be, but then again, I started off programming for money as a webmaster in grad school. I will need to dive into Dojo some more- I haven't done complex sites very recently. Dave's point about simple sites is right- and the biggest message I'm hearing from my clients- we need specialty apps that get the data across clearly.

A lot of the Q & A was taken up trying to nail home the fact that javascript is a client-side language! Once the data is transferred to the user, the server doesn't touch it unless it's uploaded back (which isn't completely out-of-the box).

ESRI Dev Summit Plenary

2008-03-18T20:53:00.000-07:00

Spatially Adjusted

davebouwman.net

Rise & Shout

Check the above for very thorough descriptions. Rather than regurgitate the same, here are some thoughts:

The javascript frameworks are going to generate a ton of new apps- it's an opportunity for any organization that has ArcGIS Server running to serve their data & have everybody else use it without messing with it.

More importantly, the web client support lessens the need to host both mapping applications & services on the same ArcGIS Server box (app vs. data).

I'll talk about it in the mashup post, but the javascript libraries look to be dead simple if you've done a mashup before (or even some moderate DHTML use & read through the google maps examples)

ArcGIS Mobile- I agree with Dave on wanting to try it out. The presentation they gave was a very good intro walkthrough.

Deployment to mobile devices is a lot easier- we can download from ArcGIS Server (makes it a lot easier to do a quick install). Even a quick "view the map on the cellphone" app would help us at this point (Parks people out in the field).

ArcGIS Explorer is maturing- I'm going to look forward to the future releases they showed.

So there's probably a reality distortion field set up in Palm Springs (the servers responded WAY too quickly), but from a programmer's standpoint, 9.3 looks to be a good release.

'Pre-conference'

2008-03-18T06:50:00.001-07:00

I'm at the ESRI Developer's Summit. Yesterday was the pre-conference. I was able to have some nice conversation with ESRI employees about ArcGIS Server; we were able to solve some issues with a web application. ESRI is naturally showing a bunch of Server 9.3 demos; they have some pretty nifty examples using the Javascript API- I'm really eager to learn more & get the beta.

The weather is gorgeous. The Business Partner Conference ended yesterday as well; it snowed for at least part of their time here. It was really interesting to talk with some of the attendees to that conference; it's a bit too bad there isn't more overlap between the two conferences.

Apparently, 'kosher-style' means...

2008-03-18T06:43:00.000-07:00

Being able to offer a ham & cheese omelet for breakfast.

I'm a ways away from NYC.