Getting to know the UK domain crawl

2014 was a fairly eventful year in Scotland; over this period, colleagues and I were working in co-operation with the British Library, National Library of Wales and Bodleian Library, to select and create archived versions of sites in two webarchive collections:

At the same time, Generation: 25 Years of Contemporary Art in Scotland exhibitions were held in more than sixty galleries across Scotland, the central exhibition being held at National Galleries of Scotland.

This further coincided with the second UK domain crawl, an approximately annual task collecting copies of websites published on the open UK web, which ran from July until December 2014. This automated crawl collects a lot of websites – could I rely on it to preserve what was published there as part of this project, now much of it is out of circulation? I have a clear memory of the relief (being new) that the domain crawl overlapped with our resources being directed elsewhere: but hadn’t taken the time to confirm whether this feeling was justified, although I am now aware that it wasn’t proportionate…  In any case, “Generation” seemed a good starting point, to explore the coverage of sites preserved by the domain crawl at the national level.

I began by trying to find relevant material in gallery websites, or failing that, a version as near to the time of the exhibition as possible, hoping to discover versions retaining the design of the originals, as in the example below – the Ingleby Gallery’s site, at a time it was communicating Generation events in its own space, the National Galleries and Perth Museum & Art Gallery:

Situated in Edinburgh and dealing in modern and contemporary art. Includes details of past current and future exhibitions and artists.

INGLEBYGALLERY.COM @ 2014-07-25T00:16:01Z
Other Versions || Original URL

I then looked for eleven venues in the domain crawl, listed in this map:
GENERATION: Venues Across Scotland… in the archive

  • UK web archive locations are listed for copies of each venue’s site, near to the time of the event as possible.
  • These links can be followed by readers in Reading Rooms in Edinburgh, or Kelvin Hall in Glasgow.

I then reviewed them for material relevant to “Generation” exibitions (see table):

Venue Generation exhibition information or article?
Cove Park Alex Frost’s project “The Patrons”
Rozelle House Galleries South by South West 25: Christine Borland, Dalziel + Scullion, Graham Fagen
Duff House no relevant material
Paxton House no relevant material
Gracefield Arts Centre South by South West 25: Christine Borland, Dalziel + Scullion, Graham Fagen
Cooper Gallery no collection at that time
Dundee Contemporary Arts Symposium: “Studio Jamming: Artists’ Collaborations in Scotland”
National Galleries of Scotland Main Generation site
Aberdeen Art Gallery Anthony Schrag: Playtime / Placetime
WASPS Artists Studios no collection at that time
Collective Ruth Ewan/Astrid Johnston: Memorialmania
Hospitalfield Arts Fieldwork, an ongoing programme of summer schools

Excluding the main Generation site, only three were actively collected during the period of the event. Seven were not: even so, most contain reference to Generation events.

My conclusion from this shallow dip into the archive is that the domain crawl did not create a complete record of Generation; but where collected, venues sometimes published much more than simple box-office information, for example results of the “Studio Jamming” writing residency published by Cooper Gallery on a dedicated site (now gone from the live web), and the related Live publishing event: “a series of free publications created, printed and disseminated in situ” and the twitter feed highlighting events and coverage.

Seeking out what remains of the Generation project through a webarchive may seem oddly indirect: these events will be well documented, books published, etc. – but learning about how the event surfaced online, by discovering the relationships between galleries and art publishing, local television and “I spy” games like [ARCHIVED] ArtHunter mobile app (a gallery crawl?), within the context provided by the archive, was interesting – and I gained a more realistic impression of the domain crawl. It isn’t everything; but more than might at first appear.

  • NB: the quoted archived copy of the Ingleby Gallery website was originally an attempt to use a method called “mementoembed“; the above is an approximation.

Eilidh MacGlone

I work in the Library's Acquisitions team, archiving Scottish websites for Modern Collections.