Temporal Expertise Profiling

0
They say that the only thing nobody can strip you off is your knowledge and experiences. And surely, it is the shared knowledge that took humanity so far. Lately, though, with the explosion of information it becomes harder and harder to find the owners of certain knowledge. And that is the reason why expertise retrieval (ER) was born as a sub-discipline of IR. It refers to the general area of linking humans to knowledge areas, and vice versa [1]. The field thrives on all the information available (online) that can be traced and mined for evidence of expertise. In the past decade, the IR community embraced it with a great deal of attention. Specifically, two main expertise retrieval tasks have been investigated:

  • expert finding : “Who are the experts on topic X?”, and
  • expert profiling : “What topics does person Y know about?”.

Related Publications

ExperTime: Tracking Expertise over Time
J. R. Benetka, K. Balog, and K. Nørvåg.
In: SIGIR ’14.
pdf / video / bib

Temporal Expertise Profiling
J. R. Benetka, K. Balog, and K. Nørvåg.
In: ECIR ’14.
pdf / bib / poster

Hermann Ebbinghaus (1850 – 1909) pioneered the experimental study of memory, and is known for his discovery of the forgetting curve and the spacing effect.

In our paper, we focus on the expert profiling task with the ultimate goal of identifying and characterizing changes in expertise of individuals over time. The temporal aspect, although mostly ignored in ER, has a far-reaching impact on one’s current state of knowledge and her ways of reasoning. If we ignore the obvious, i.e., that expertise fades away with time if not practiced (e.g., Ebbinghaus’s experiment), there might be an even more fundamental connotation which is that the order of topics we learn actually influences the outcome [2]. Simply, a mathematician-turned-biologist doesn’t necessarily think the same way as biologist-turned-mathematician. Our approach to include the temporal dimension to expertise profiling is based on four components:

  1. Creation of expertise profile.
  2. Hierarchical representation of expertise profile.
  3. Temporal expertise profile.
  4. Detecting changes in profiles.

The process of temporal expertise profiling: from creation of a hierarchical expertise profile, through iteration over temporal snapshots, to detection of changes.

Creation of expertise profile

In [3] the topical profile of an individual is defined as “a record of the types and areas of skills and knowledge of that individual, together with an identification of levels of ‘competency’ in each.” Following prior work [1], we estimate expertise based on a set of documents authored by the person. Given a set of topics of interest \(C={c_1,\dots, c_n}\), the expertise profile is defined as a vector of weights computed for each of the topics \(W={w_1,…, w_n}\). Assuming that we can associate each document \(d\) (e.g., by creating language models) with one or multiple topics \(c\) with probability \(P(c|d)\), we can proportionally distribute the document’s weight to the corresponding topics in the weight vector (i.e., expertise profile).

Hierarchical representation

Contrary to the horizontal structure of the weight vector described before, the hierarchical expertise profile is defined as a weighted tree that is built around a topical taxonomy (rather than a flat list of topics). A node in the tree corresponds to one topic (e.g., Psychology), its child node represents a sub-topic (e.g., Neuropsychology). Given a document pertinent to certain topic (i.e., evidence of expertise), its weight is distributed in a bottom-up fashion, starting with the leaf nodes and then propagating weights to the upper levels until the root of the tree is reached. The weights of non-leaf nodes are therefore sum of the weights of direct descendants. The major benefit of such hierarchical organization is in the ability to operate on arbitrary abstraction level of one’s expertise profile.

Example hierarchical expertise profile, constructed from documents shown on the left. Node sizes are set proportional their weight (note that edges are not weighted according to our definition, thickness is only applied here for presentation purposes).

Temporal expertise profile

We define the temporal expertise profile of a person as a series of hierarchical expertise profiles computed at different points in time. The time interval between the points may be regular, such as one year, as well as non-regular. We refer to each partial profile created within the period between two temporal points as the profile snapshot and it is a combination of two components:

  • 1) expertise acquired in the corresponding time period, and
  • 2) expertise “carried over” from the past.

The first component is computed the same way as in atemporal profiling with the only difference that we restrict ourselves to documents originating from the given time period (e.g., year 2017). As for the second part, we apply a decay function onto past expertise profile to imitate the notion of expertise “fading away” over time.

Progression of expertise profile in time.


Detecting changes

Let’s say we have an expertise profile and we wish to compare its progression between two points in time. For each of the two temporal snapshots, we pin down a single node or small set of nodes that accumulate the majority of the node weights and we compare dissimilarities in these focus nodes, as we call them. We then characterize changes in the profile based on the position in hierarchy where the change occurs. We differentiate between changes of the entire field of expertise if the focus node moves from one top-level node to another, and changes in the topics of expertise if the change appears on lower levels of the hierarchy.

We divide changes in one’s expertise into changes on the level of research fields and topics of research.



References

[1] Expertise retrieval., K. Balog et al., Foundations and Trends in Information Retrieval, 2012.
[2] In Order to Learn: How the sequence of topics influences learning., F. E. Ritter et al., Oxford Scholarship Online, 2010.
[3] Determining expert profiles (with an application to expert finding)., K. Balog and M. de Rijke, Proceedings of IJCAI ’07, 2007.