Xenoarchaeology is the study of past alien cultures from their physical remains.  The prefix xeno- is from the Greek xenos, ‘stranger’.

Alien, in this instance, refers to members of any species other than that of the hypothetical xenoarchaeologist.  A human studying martian ruins is a xenoarchaeologist, as is a martian studying human ruins.

The term ‘alien’ always sounds a little pejorative to me (how about non-human person?), but I use it here instead of ‘extraterrestrial’ which could be taken as a spatial designation, and because in the future there could conceivably be terrestrial nonhuman cultures (say, from uplifted animals or artificial intelligences). Despite its connotations, the word ‘alien’ conveys the otherness of the culture to be studied.

If xenoarchaeology is the study of past cultures from species other than one’s own, and if you define ‘human’ in a narrow sense to refer to modern Homo sapiens, then archaeologists who study other hominids are already conducting a sort of xenoarchaeology.

Interpreting human, and perhaps hominid, minds and cultures is one thing, in fact we all do it in our everyday lives. But a special set of skills will be needed to study alien cultures without anthropomorphising them.

See Robert Freitas’ paper Xenopsychology or this article on Astrosociobiology for some reading on studying alien cultures.

I’m put in mind of a concept from the Ender’s Game series by science fiction author Orson Scott Card. In it, he introduces a system of classifying ‘strangers’ called the Hierarchy of Exclusion. Here’s an abridged version from Wikipedia:

• Utlanning (translated: “outlander” or “foreigner””, utlänning in Swedish) are strangers of one’s own species and one’s own culture). An utlanning is a person who shares the observer’s cultural identity.
• Framling (translated: “stranger”, främling in Swedish) are members of one’s own species but from another culture. This is a person who is both substantially similar to and significantly different from ourselves.
• Ramen are strangers from another species who are capable of communication and peaceful coexistence.
• Varelse (pronounced var-ELSS-uh) (translated: “being”) are strangers from another species who are not able to communicate with us. They are true aliens, completely incapable of common ground with humanity. In Swedish, varelse means “creature.”
• Djur (translated as: “slavering beast”): are the monsters. “The dire beast that comes in the night with slavering jaws.” Translated from Swedish, djur means “animal”.

Archaeologists today study utlanning and framling remains. Xenoarchaeologists will study ramen and varelse. I don’t have the books to hand, so I’m not sure whether djur refers literally to animals/monsters or to implacably hostile aliens.

Ramen would include most of the aliens of popular science fiction – the Star Trek and Star Wars universes, for example. For examples of varelse consider the aliens from John Varley’s The Ophiuchi Hotline, Peter Watts’ Blindsight, or the creators of the artefact from Algis Budrys’ Rogue Moon.

It seems to me that the distinction between ramen and varelse is potentially a useful one, at the very least offering shorthand terms for use in discussing aliens – fictional or otherwise.