We illustrate the use of LQL by applying it to an important language analysis problem for bioscience text: noun compound (NC) bracketing. Consider the phrases liver cell antibody and liver cell line. Although equivalent at the part of speech (POS) level, they have different syntactic trees. In the former case, an antibody targets a liver cell, while in the latter we talk about a cell line which is derived from the liver. The distinction can be represented as a binary tree or, equivalently, as a binary bracketing:
[ [ liver cell ] antibody ] (left bracketing)
[ liver [cell line] ] (right bracketing)
There is little prior work on NC bracketing. Best known is that of Lauer (1995) who introduces the probabilistic dependency model for the syntactic disambiguation of NCs and argues against the adjacency model, proposed in (Pustejovsky et al., 1993). The probabilistic dependency model compares Pr(w1|w3) to Pr(w2|w3). Here Pr(wi|wj) means the probability that wi precedes wi, for a given wj. The adjacency model compares Pr(w1|w2) to Pr(w2|w3). Lapata and Keller (2004) propose using the Web as a baseline with an application to six NLP tasks, including the syntactic disambiguation of NCs, and show that variations on bigram counts perform nearly was well as more elaborate methods.
However, the use of Web search engines imposes limitations on what kinds of queries we can write, mainly because of the lack of linguistic annotation. For example, if we want to estimate the probability that health precedes care #("health care")/#(care) , we need the frequencies of "health care" and care, where both words are nouns. The problem is that a query for care will return many pages where it is used as a verb, while health care would nearly always occur as a noun. Even when both health and care are used as nouns and are adjacent, they may belong to different NPs but sit next to each other only by chance. Furthermore, since search engines ignore punctuation characters, the two nouns may also come from different sentences.
Other Web search engine restrictions prevent querying directly for terms containing hyphens or possessive markers such as amino-acid sequence and brain stem’s cells. They also disallow querying for a term like bronchoalveolar lavage (BAL) fluid, in which the internal parenthesized abbreviation suggests a left bracketing. Finally, search engines cannot support queries that make use of generalized POS information such as
in which the uppercase patterns stand for any verb and any preposition. Further, using page hits as a proxy for n-gram frequencies can produce some counter-intuitive results. Consider the bigrams w1w4, w2w4 and w3w4 and a page that contains each bigram exactly once. A search engine will contribute a page count of 1 for w4 instead of a frequency of 3; thus the page hits for w4 can be smaller than the page hits for the sum of the individual bigrams. See Keller and Lapata (2003) for more details.
Below we describe the use of LQL and architecture to acquire statistics for language phenomena that make use of NLP results. These examples include using POS and shallow-parse annotation layers. The system also supports queries on other kinds of annotation layers not shown here, such as gene/protein names, MeSH labels, and abbreviation expansions.
The first thing we need is a set of noun compounds to be further split into a training and a testing sets. For the purpose, we could write the following LQL query:
FROM
[layer='shallow_parse' && tag_name='NP'
^ [layer='pos' && tag_name="noun"]
[layer='pos' && tag_name="noun"]
[layer='pos' && tag_name="noun"] $
] AS compound
SELECT compound.content
View sample results here.
It looks for a noun phrase (NP) from the shallow parse layer, containing exactly three nouns (from the POS layer). Each layer in LQL is enclosed in brackets and is optionally followed by a binding statement. Layers have names, e.g. pos, shallow_parse, which further determine a possible set of types such as NP for shallow_parse and NN for pos. There are macros for groups of tags, such as noun, which refers to all POS which are nouns in the Penn Treebank. Single quotes are used for exact matches, and double for case insensitive ones and macros. When a layer spans another one, the latter is enclosed inside the former. By default, layers immediately follow each other, as the three nouns above. If we further want to express that no other words are allowed before the first or after the last noun, we also need the UNIX delimiters ^ and $. Finally, there is a select statement specifying what needs to be returned.
The above query is a good start but we might want to refine it. For example, we might want to focus on the most frequent NCs. For the purpose, we can enclose the LQL inside an SQL block and then use SQL aggregation functions to produce a sorted list (in descending order) of the NCs and their corresponding frequencies:
SELECT lql.compound compound,
COUNT(*) AS freq
FROM (
BEGIN_LQL
FROM
[layer='shallow_parse' && tag_name='NP'
^ [layer='pos' && tag_name="noun"]
[layer='pos' && tag_name="noun"]
[layer='pos' && tag_name="noun"] $
] AS compound
SELECT compound.content AS compound
END_LQL
) AS lql
GROUP BY lql.compound
ORDER BY freq DESC
There is still a problem with that query: it is case sensitive. This might be good in the biomedical domain, where the naming of genes, proteins etc. sometimes cares about capitalization. But when this is not the case, the query above partitions the occurrences of the same NP into several different surface forms. We could fix this by forcing the noun compounds to lower case:
SELECT LOWER(lql.compound) compound,
COUNT(*) AS freq
FROM (
BEGIN_LQL
FROM
[layer='shallow_parse' && tag_name='NP'
^ [layer='pos' && tag_name="noun"]
[layer='pos' && tag_name="noun"]
[layer='pos' && tag_name="noun"] $
] AS compound
SELECT compound.content AS compound
END_LQL
) AS lql
GROUP BY LOWER(lql.compound)
ORDER BY freq DESC
Now, let us look closer inside the NP. While this query will only extract three-word NCs, provided that the POS tagging and the shallow parsing annotations are correct, it will also miss some. The problem is that we have been too restrictive by requiring that the NP should contain nothing but three nouns. While this rules out the possibility for e.g. a four-word NC, it also does not allow words with other legitimate POS, such as determiners (e.g. the, a, an, this), adjectives etc. It would be better to assert that the three nouns should occupy the three last positions in the NP and to disallow other nouns within the same NP. We can achieve this by using a layer negation and artificial ranges:
SELECT LOWER(compound.content) lc,
COUNT(*) AS freq
FROM
BEGIN_LQL
FROM
[layer='shallow_parse' && tag_name='NP'
^ ( { ALLOW GAPS }
![layer='pos' && tag_name="noun"]
(
[layer='pos' && tag_name="noun"]
[layer='pos' && tag_name="noun"]
[layer='pos' && tag_name="noun"]
) $
) $
] AS compound
SELECT compound.content
END_LQL
GROUP BY lc
ORDER BY freq DESC
The last version of the query looks quite complicated. First, we have an assertion that a layer does not exist, which is achieved by preceding it by ! (an exclamation mark). We further need to say that the non-existing layer is disallowed anywhere before the sequence of three nouns, not just when it immediately precedes the first one. For the purpose, we introduce the outer artificial range and we specify that gaps (i.e. any intermediate words) are allowed between any layers it contains. Now, we need one more artificial layer around the three nouns to disallow gaps between them.
Finally, we might want to have the individual words separately, rather than as a single string.
SELECT LOWER(word1.content) lc1,
LOWER(word2.content) lc2,
LOWER(word3.content) lc3,
COUNT(*) AS freq
FROM
BEGIN_LQL
FROM
[layer='shallow_parse' && tag_name='NP'
^ ( { ALLOW GAPS }
![layer='pos' && tag_name="noun"]
(
[layer='pos' && tag_name="noun"] AS word1
[layer='pos' && tag_name="noun"] AS word2
[layer='pos' && tag_name="noun"] AS word3
) $
) $
]
SELECT word1.content, word2.content, word3.content
END_LQL
GROUP BY lc1, lc2, lc3
ORDER BY freq DESC
Now we can select a subset of the noun compounds and annotate them with information about whether the bracketing is left, right, both or neither. After discarding the last group, we obtain an annotated set, which will be further split into training and testing subsets. For both sets, we need to extract some features that could help us to predict the correct bracketing. These include:
Below we show some sample queries for extracting bigram counts and paraphrases (prepositional, copula and verbal).
SELECT COUNT(*) AS freq
FROM
BEGIN_LQL
FROM
[layer='sentence'
[layer='pos' && content="growth"] AS word1
[layer='pos' && content="factor"]
]
SELECT word1.content
END_LQL
This query will count the word bigrams, respecting the sentence boundaries, but will not require them to be nouns, just to be consecutive. Note that we did not need to select both words, just one of them, as all we are interested in is the total count. If we want both words to be nouns, we can write:
SELECT COUNT(*) AS freq
FROM
BEGIN_LQL
FROM
[layer='sentence'
[layer='pos' && tag_name="noun" && content="growth"] AS word1
[layer='pos' && tag_name="noun" && content="factor"]
]
SELECT word1.content
END_LQL
And we can further restrict them to be inside the same NP. This will filter out the occurrences that happened to be together by chance. Here is the query:
SELECT COUNT(*) AS freq
FROM
BEGIN_LQL
FROM
[layer='shallow_parse' && tag_name='NP'
[layer='pos' && tag_name="noun" && content="growth"] AS word1
[layer='pos' && tag_name="noun" && content="factor"]
]
SELECT word1.content
END_LQL
We might want to allow for inflections of the bigram. This means to allow for inflection of the second word:
SELECT COUNT(*) AS freq
FROM
BEGIN_LQL
FROM
[layer='shallow_parse' && tag_name='NP'
[layer='pos' && tag_name="noun" && content="growth"] AS word1
[layer='pos' && tag_name="noun" && (content="factor" || content="factors")]
]
SELECT word1.content
END_LQL
We also need the unigram counts for a word and its inflections. This is a fairly simple query:
SELECT COUNT(*) AS freq
FROM
BEGIN_LQL
FROM
[layer='pos' && tag_name="noun" && (content="factor" || content="factors")] AS word
SELECT word.content
END_LQL
Paraphrases are an important feature type for determining NC bracketing. For example, an author describing the concept of human immunodeficiency virus may choose to write it in a more expanded manner, such as immunodeficiency virus in humans. If this prepositional paraphrase occurs often, it suggests that the full NC has a right bracketing, since immunodeficiency and virus are kept together in the expanded version. This term is ambiguous though and can be written also as virus of the human immunodeficiency (left bracketing).
There are NCs whose meaning cannot be expressed with a paraphrase at all, or at least not with a prepositional one (Warren, 1978). In the latter case, one could try a copula paraphrase like immunodeficiency virus that/which is human (right bracketing), or a verb paraphrase such as virus causing human immunodeficiency (left) or immunodeficiency virus found in humans (right).
Instead of trying to manually decide the correct paraphrases, we can issue queries on the appropriate paraphrase patterns and find out how often each occurs in the corpus. We then add up the number of hits predicting a left versus a right bracketing and compare the counts to make a decision.
Here we show a sample query looking for left bracketing predictions, based on prepositional paraphrases (for the NC nerve growth factor):
SELECT COUNT(*) AS freq
FROM
BEGIN_LQL
FROM
[layer='sentence'
[layer='pos' && content="factor"]
[layer='pos' && content="for"] AS prep
[layer='pos' && content="nerve"]
[layer='pos' && content="growth"]
]
SELECT prep.content
END_LQL
The query above does not checks for POS, but we can easily do so:
SELECT COUNT(*) AS freq
FROM
BEGIN_LQL
FROM
[layer='sentence'
[layer='pos' && tag_name="noun" && content="factor"]
[layer='pos' && tag_name='IN' && content="for"] AS prep
[layer='pos' && tag_name="noun" && content="nerve"]
[layer='pos' && tag_name="noun" && content="growth"]
]
SELECT prep.content
END_LQL
We might also want to allow for an optional determiner, such as the definite article the, after the preposition. We can do so by inserting a layer preceded by a question mark:
SELECT COUNT(*) AS freq
FROM
BEGIN_LQL
FROM
[layer='sentence'
[layer='pos' && tag_name="noun" && content="factor"]
[layer='pos' && tag_name='IN' && content="for"] AS prep
?[layer='pos' && tag_name='DT' && content="the"]
[layer='pos' && tag_name="noun" && content="nerve"]
[layer='pos' && tag_name="noun" && content="growth"]
]
SELECT prep.content
END_LQL
Further, we can check for several prepositions and determiners simultaneously getting the counts for the different prepositions separately and in descending order, allowing also for inflections, e.g.:
SELECT LOWER(prep.content) lp,
COUNT(*) AS freq
FROM
BEGIN_LQL
FROM
[layer='sentence'
[layer='pos' && tag_name="noun" && (content="factor" || content="factors")]
[layer='pos' && tag_name='IN' &&
(content="of" || content="for" || content="in" || content="at" ||
content="on" || content="from" || content="with" ||
content="about" || content="like")
] AS prep
?[layer='pos' && tag_name='DT' &&
(content="the" || content="a" || content="an")
]
[layer='pos' && tag_name="noun" && content="nerve"]
[layer='pos' && tag_name="noun" && (content="growth" || content="growths")]
]
SELECT prep.content
END_LQL
GROUP BY lp
ORDER BY freq DESC
We can rewrite this query without the optional layer:
SELECT LOWER(prep.content) lp,
COUNT(*) AS freq
FROM (
SELECT prep.content
FROM
BEGIN_LQL
FROM
[layer='sentence'
[layer='pos' && tag_name="noun" && (content="factor" || content="factors")]
[layer='pos' && tag_name='IN' &&
(content="of" || content="for" || content="in" || content="at" ||
content="on" || content="from" || content="with" ||
content="about" || content="like")
] AS prep
[layer='pos' && tag_name="noun" && content="nerve"]
[layer='pos' && tag_name="noun" && (content="growth" || content="growths")]
]
SELECT prep.content
END_LQL
UNION ALL
SELECT prep.content
FROM
BEGIN_LQL
FROM
[layer='sentence'
[layer='pos' && tag_name="noun" && (content="factor" || content="factors")]
[layer='pos' && tag_name='IN' &&
(content="of" || content="for" || content="in" || content="at" ||
content="on" || content="from" || content="with" ||
content="about" || content="like")
] AS prep
[layer='pos' && tag_name='DT' &&
(content="the" || content="a" || content="an")
]
[layer='pos' && tag_name="noun" && content="nerve"]
[layer='pos' && tag_name="noun" && (content="growth" || content="growths")]
]
SELECT prep.content
END_LQL
)
GROUP BY lp
ORDER BY freq DESC
It would be probably better, if we write a more general query, which allows any determiner and any preposition:
SELECT LOWER(prep.content) lp,
COUNT(*) AS freq
FROM
BEGIN_LQL
FROM
[layer='sentence'
[layer='pos' && tag_name="noun" && (content="factor" || content="factors")]
[layer='pos' && tag_name='IN']
?[layer='pos' && tag_name='DT']
[layer='pos' && tag_name="noun" && content="nerve"]
[layer='pos' && tag_name="noun" && (content="growth" || content="growths")]
]
SELECT prep.content
END_LQL
GROUP BY lp
ORDER BY freq DESC
Of course, we could write more sophisticated queries, as the one below, which combines the last query with what we had in NC4, allowing for more modifiers than just a single determiner, but also requiring that the words are the last in some noun phrases (but disallowing other nouns in these NPs):
SELECT LOWER(prep.content) lp,
COUNT(*) AS freq
FROM
BEGIN_LQL
FROM
[layer='sentence'
[layer='shallow_parse' && tag_name='NP'
^ ( { ALLOW GAPS }
![layer='pos' && tag_name="noun"]
[layer='pos' && tag_name="noun" && (content="factor" || content="factors")] $
) $
]
[layer='pos' && tag_name='IN'] AS prep
[layer='shallow_parse' && tag_name='NP'
^ ( { ALLOW GAPS }
![layer='pos' && tag_name="noun"]
(
[layer='pos' && tag_name="noun" && content="nerve"]
[layer='pos' && tag_name="noun" && (content="growth" || content="growths")] $
) $
) $
]
]
SELECT prep.content
END_LQL
GROUP BY lp
ORDER BY freq DESC
Probably the last query is too complicated and NC_PARAPHRASE_PREPOSITIONAL_4 and NC_PARAPHRASE_PREPOSITIONAL_5 are better. They are not that restrictive and do not rely that much on a correct shallow parse.
Finally, if we want to write a right-predicting rule, all we need is to move the second word (in our example this is growth):
SELECT LOWER(prep.content) lp,
COUNT(*) AS freq
FROM
BEGIN_LQL
FROM
[layer='sentence'
[layer='pos' && tag_name="noun" && content="growth"]
[layer='pos' && tag_name="noun" && (content="factor" || content="factors")]
[layer='pos' && tag_name='IN' &&
(content="of" || content="for" || content="in" || content="at" ||
content="on" || content="from" || content="with" ||
content="about" || content="like")
] AS prep
?[layer='pos' && tag_name='DT' &&
(content="the" || content="a" || content="an")
]
[layer='pos' && tag_name="noun" && (content="nerve" || content="nerves")]
]
SELECT prep.content
END_LQL
GROUP BY lp
ORDER BY freq DESC
We can look for copula paraphrases using a query similar to NC_PARAPHRASE_PREPOSITIONAL_LEFT_4. This time we are interested in expressions of the kind growth factor that is nerve (the example is for a left bracketing though, i.e. for factor that is nerve growth, which sounds a little bit odd):
SELECT COUNT(*) AS freq
FROM
BEGIN_LQL
FROM
[layer='sentence'
[layer='pos' && tag_name="noun" && (content="factor" || content="factors")] AS word1
[layer='pos' && (content="that" || content="which" || content="who")]
[layer='pos' && (content="is" || content="are")]
?[layer='pos' && tag_name='DT' &&
(content="the" || content="a" || content="an")
]
[layer='pos' && tag_name="noun" && content="nerve"]
[layer='pos' && tag_name="noun" && (content="growth" || content="growths")]
]
SELECT word1.content
END_LQL
Again, if we want to write a right-predicting rule, all we need is to move growth:
SELECT COUNT(*) AS freq
FROM
BEGIN_LQL
FROM
[layer='sentence'
[layer='pos' && tag_name="noun" && content="growth"] AS word1
[layer='pos' && tag_name="noun" && (content="factor" || content="factors")]
[layer='pos' && (content="that" || content="which" || content="who")]
[layer='pos' && (content="is" || content="are")]
?[layer='pos' && tag_name='DT' &&
(content="the" || content="a" || content="an")
]
[layer='pos' && tag_name="noun" && (content="nerve" || content="nerves")]
]
SELECT word1.content
END_LQL
Here we look for paraphrases of the kind factor associated with the nerve growth, i.e. we want any verb (or we could insert a specific verb or set of verbs), followed by an optional preposition and an optional determiner. We will write a query similar to NC_PARAPHRASE_PREPOSITIONAL_LEFT_4 and NC_PARAPHRASE_COPULA_LEFT_4:
SELECT COUNT(*) AS freq
FROM
BEGIN_LQL
FROM
[layer='sentence'
[layer='pos' && tag_name="noun" && (content="factor" || content="factors")]
[layer='pos' && tag_name="verb"] AS verb
?[layer='pos' && tag_name='IN']
?[layer='pos' && tag_name='DT']
[layer='pos' && tag_name="noun" && content="nerve"]
[layer='pos' && tag_name="noun" && (content="growth" || content="growths")]
]
SELECT verb.content
END_LQL
And again, if we want to write a right-predicting rule, all we need is to move growth:
SELECT COUNT(*) AS freq
FROM
BEGIN_LQL
FROM
[layer='sentence'
[layer='pos' && tag_name="noun" && content="growth"]
[layer='pos' && tag_name="noun" && (content="factor" || content="factors")]
[layer='pos' && tag_name="verb"] AS verb
?[layer='pos' && tag_name='IN']
?[layer='pos' && tag_name='DT']
[layer='pos' && tag_name="noun" && (content="nerve" || content="nerves")]
]
SELECT verb.content
END_LQL
Without optional layers the last query above can be re-written as:
SELECT COUNT(*) AS freq
FROM (
SELECT verb.content
FROM
BEGIN_LQL
FROM
[layer='sentence'
[layer='pos' && tag_name="noun" && content="growth"]
[layer='pos' && tag_name="noun" && (content="factor" || content="factors")]
[layer='pos' && tag_name="verb"] AS verb
[layer='pos' && tag_name="noun" && (content="nerve" || content="nerves")]
]
SELECT verb.content
END_LQL
UNION ALL
SELECT verb.content
FROM
BEGIN_LQL
FROM
[layer='sentence'
[layer='pos' && tag_name="noun" && content="growth"]
[layer='pos' && tag_name="noun" && (content="factor" || content="factors")]
[layer='pos' && tag_name="verb"] AS verb
[layer='pos' && tag_name='IN']
[layer='pos' && tag_name="noun" && (content="nerve" || content="nerves")]
]
SELECT verb.content
END_LQL
UNION ALL
SELECT verb.content
FROM
BEGIN_LQL
FROM
[layer='sentence'
[layer='pos' && tag_name="noun" && content="growth"]
[layer='pos' && tag_name="noun" && (content="factor" || content="factors")]
[layer='pos' && tag_name="verb"] AS verb
[layer='pos' && tag_name='DT']
[layer='pos' && tag_name="noun" && (content="nerve" || content="nerves")]
]
SELECT verb.content
END_LQL
UNION ALL
SELECT verb.content
FROM
BEGIN_LQL
FROM
[layer='sentence'
[layer='pos' && tag_name="noun" && content="growth"]
[layer='pos' && tag_name="noun" && (content="factor" || content="factors")]
[layer='pos' && tag_name="verb"] AS verb
[layer='pos' && tag_name='IN']
[layer='pos' && tag_name='DT']
[layer='pos' && tag_name="noun" && (content="nerve" || content="nerves")]
]
SELECT verb.content
END_LQL
)