Shift-reduce recognition

Let’s consider one particular way in which sentence recognition could proceed in accord with the phrase-structure rules given above (where “in accord with” is deliberately agnostic, for now, about the distinction between grammar-as-abstraction and grammar-as-information): shift-reduce recognition.

There are two kinds of moves, or transitions, that this simple kind of processor can make. A shift transition consumes a word of input and pushes the word’s category onto the stack that comprises the processor’s memory. A reduce transition replaces two adjacent symbols on the top of the stack, each of which represents the category of some constituent that has been found, with another symbol representing the category of their parent constituent.

Here’s how this kind of processor would recognize the sentence the dog chased it.

Suppose we have determined that the sequence of steps shown in this table is an accurate characterization of the steps that are carried out in the relevant speakers’ minds in the course of comprehending that sentence. We can then consider the different ways of thinking about grammatical rules: in each line of the table above, what is the relationship between the rule listed in the third column and the stack-manipulation that happens in the fifth column?

According to the grammar-as-abstraction view, the processor has transitions available to it that manipulate the stack in certain ways – for example, it can consume the word dog and push N onto the stack, and it can replace a V NP sequence at the top of the stack with a VP – and those stack-manipulating transitions are, in a sense, “all there is”. Writing things like VP → V NP in the third column is a helpful way for us to encapsulate certain aspects of what’s going on (and writing out the full rule list at the beginning of this post is a helpful way to summarize the overall capacity of this processing device), but no more than that.

According to the grammar-as-information view, on the other hand, the rules are more fundamental than the stack manipulations. On this view there is a mental list of rules which is consulted, and the fact that the processor finds VP → V NP in that list is what “tells it” that it can replace V NP with VP. On each line of the table, the rule in the third column is what the processor has looked at in order to determine that the stack manipulations in the fifth column are licensed. The processor, on this view, is something that takes in two different kinds of information, a collection of grammar rules and a sequence of words, and tries to knit them together in a certain way.

In this particular case of a simple shift-reduce processor, I think it is very difficult to tell these two views apart. This is because there is a one-to-one correspondence between processing transitions and grammar rules. For every shift transition that the grammar-as-abstraction proponent posits as part of the processor, there is exactly one rule of the form X → blah that the grammar-as-information proponent posits on the list of rules; and for every reduce transition, there is exactly one rule of the form X → Y Z. Knowing that the relevant mental processing proceeds as in the table above seems equally compatible with either of these views.

If this was the situation we were in regarding human sentence processing, perhaps one could argue that, in the absence of positive evidence for the kind of separation that the grammar-as-information view posits, the grammar-as-abstraction view should be preferred on the grounds of simplicity; or, perhaps one could argue that, because of the one-to-one correspondence between transitions and rules, the supposed distinction between the two views turns out to be misguided, and the question is moot. But this is probably not actually the situation we find ourselves in when it comes to human sentence processing: the relevant mechanisms there are, to the best of our knowledge, more complicated than the simple shift-reduce system, and more complicated in ways that allow for the distinction between the two views to come more clearly into focus.

Left-corner recognition

Things get more interesting when we turn to left-corner recognition. The fundamental difference compared to shift-reduce recognition is that the processor is able to represent a certain kind of incomplete constituents. More specifically, the stack maintained by a left-corner recognizer contains not just symbols like NP and VP representing constituents that have been completely identified, but also symbols of the form X/Y (e.g. NP/N or S/VP) which represent an X that is missing a Y on its right edge.

Here are the steps by which a left-corner recognizer would process the same sentence from before.

To clarify the meaning of the X/Y notation, here are pictures representing the first three steps.

Unlike the shift-reduce system, where the NP constituent is posited only after both of its daughter constituents are identified, here the NP constituent is posited at Step 2, when only its left daughter has been identified; the transitions of this form are called project transitions. Of course the fact that the right daughter hasn’t yet been found needs to be tracked, which is why after Step 2 we have NP/N on the stack rather than simply NP. At Step 3 the processor consumes the next word, dog, which allows it to “pay off the debt” and arrive at a full NP; this kind of debt-repayment transition is called a match transition (since it “matches” the owed category against the incoming word).

The project transition at Step 4 introduces the S constituent, on the basis of having found its left daughter, in the same way that the one at Step 2 introduced the NP constituent. Notice that at Step 5, the incoming word chased cannot itself pay off the owed VP, so this is not a match transition like the one at Step 3, but rather is a shift transition (like the one at Step 1) which adds to the stack, specifically it introduces a V.

The next transition is where things get particularly interesting.

The transition at Step 6 is different from any of the three kinds of transitions we have seen before (shift, project and match). It has something in common with the project transitions that we saw at Step 2 and Step 4: it involves positing a VP on the basis of its left daughter (a V), and taking on a debt of the right daughter (an NP). But instead of this posited parent constituent being freshly introduced to the stack, like the NP at Step 2 or the S at Step 4, this posited parent is used to pay off the owed VP. The newly-owed NP becomes all that is missing from the S constituent, hence we have simply S/NP. This kind of transition is called a complete transition. The final match transition at Step 7 fills in the missing NP (in just the same way that the match transition at Step 3 filled in a missing N).

What I take to be significant here is that, unlike in the simple shift-reduce system, there is no one-to-one correspondence between grammatical rules and stack-manipulating transitions: for each grammatical rule of the form X → ..., the processor has one transition that introduces a new (possibly incomplete) X, and one transition that pays off an owed X. For rules of the form X → blah, this is the distinction between what shift and match (both of which consume blah from the input) do with X. For rules of the form X → Y Z, this is the distinction between what project and complete (both of which build on a Y and take on an owed Z) do with X.

So what?

It’s quite easy to make sense of this under the grammar-as-information view. The processor itself “knows” only the four schemas or templates illustrated above (one each for shift, match, project and complete), and consults the list of grammatical rules to determine how those schemas can be instantiated. The transition at Step 2 above arose from instantiating the project schema using the rule NP → D N, and the transition at Step 6 arose from instantiating the complete schema using the rule VP → V NP.

In the example above, we don’t see any single grammatical rule being used by two distinct schemas, but if we modify the sentence very slightly we can see what this would look like: if the sentence were the dog chased the cat, then the first six steps would proceed as above, and then at Step 8 we see a complete step that makes use of the same NP → D N rule that was used to instantiate the project schema at Step 2.

So the rule NP → D N can be used to instantiate the project schema to yield a transition that replaces D with NP/N, and can also be used to instantiate the complete schema to yield a transition that replaces ⋯/NP D with ⋯/N.

More generally, given some list of rules, each one is going to be instantiable via two schemas (either by shift and by match, or by project and by complete), and thus will give rise to two possible transitions. If there are, say, 10 rules in the grammar, the four schemas above will give rise to 20 possible transitions.

This makes things start to look strange under the grammar-as-abstraction view. If the stack-manipulating transitions are primitive, rather than derived by instantiating schemas, then there is no particular reason to think that these transitions should come in the neat pairs that they seem to. It would look like a coincidence if, for every transition that replaces Y with X/Z, there’s always another transition that replaces ⋯/X Y with ⋯/Z. If the transition that replaces D with NP/N and the transition that replaces ⋯/NP D with ⋯/N are just two of the 20 primitive transitions, then “the next language over” might have just one of these two; this would mean that the constituents that could appear as an NP in a subject position would differ from those that could appear as an NP in an object position, for example. To the extent that the next language over always corresponds to what we would get by making some change to the underlying list of rules and letting the transition schemas work with that modified list – for example, if the next language over lacks both the D to NP/N stack transition and the ⋯/NP D to ⋯/N stack transition – it makes less sense to think of the transitions themselves as primitive.

Notice how this is different from the situation with shift-reduce recognition, where there’s a one-to-one correspondence between rules and transitions. If the only consequence of having NP → D N in your list of rules is the possibility of stack transitions that replace D N with NP, then it matters less whether you think of the rule or the transition as primitive. From a grammar-as-information perspective, the next language over might lack the NP → D N rule, and therefore the processor won’t have available the move of replacing D N with NP; this is about the same as the prediction we’d make from a grammar-as-abstraction perspective, where the next language over just lacks the primitive stack transition that replaces D N with NP.

Another way to make the point is to observe that, if I tell you that the tree diagram for a certain sentence includes an NP node with a D left daughter and an N right daughter, you know that the shift-reduce recognition of this sentence will necessarily involve a step of replacing D N with NP. But we can’t say (without knowing more about the position of the NP node) whether the left-corner recognition of this sentence will involve a step of replacing D with NP/N, or a step of replacing ⋯/NP D with ⋯/N. This one-to-many (specifically, one-to-two) relationship between the rules and transitions is what provides the impetus for treating a rule itself as a meaningful primitive, something that can be separated from any single action that might take place in the course of processing a sentence.

The bigger picture

The main suggestion I want to put on the table is that this kind of one-to-many relationship between grammar rules and parser actions is a key form of evidence that might be used in support of the grammar-as-information view. One might agree that this is an interesting form of argument that could in principle be used to in support of the grammar-as-information view, without buying into any claims that the specifics of human sentence processing look anything like the picture I laid out with left-corner recognition above.

Having said that, there’s some reason to think that human processing does in fact proceed in something along the lines of the left-corner manner outlined above. The evidence is based on the fact that the amount of memory (stack space) required by the left-corner recognizer gets large for center-embedded sentences, but not for left-embedded or right-embedded sentences, and therefore aligns neatly with the pattern of difficulty that humans experience with these constructions (Resnik, 1992, and many references there). And as Resnik points out in that paper (section 4), it’s particularly critical that when a posited parent node is going to “pay off a debt”, this must happen as soon as the parent is posited; in other words, it’s critical that we have complete as a separate transition schema from project, which is what leads to the one-to-many relationship between rules and transitions. (The alternative “arc-standard” system where rules of the form X → Y Z are only associated with project transitions, and there’s a separate operation that “cancels out” parents posited by project with owed categories, does not yield the same predictions about memory load.)

Of course, a full-fledged theory of human sentence processing is going to look different in the details, not least because the relevant notion of syntactic structure-building is likely to have more bells and whistles than the simple trees that are describable via phrase-structure rules of the sort used above. If the processing system needs to deal with those additional bells and whistles, is it likely that things will fall neatly into place with a one-to-one correspondence between rules (of a new enriched sort) and transitions, like in the shift-reduce system, or it is more likely that things will get yet more complicated and diverge even further from such a one-to-one correspondence? It doesn’t seem crazy to lean slightly towards the latter, all else being equal. And for what it’s worth, although I’m not suggesting that this should hold much water, this seems to be what we see in parsers that extend the left-corner system illustrated above to incorporate movement (Hunter et al., 2019; Stanojević & Stabler, 2018): the point in the above was that the parser needs to treat merge steps that create left daughters differently from those that create right daughters, and once movement is introduced it becomes similarly necessary to distinguish typical, simple movement dependencies from those that have been derived via remnant movement.

A little bit about our point of view

We are all trained and working in the generative grammar tradition and consider ourselves, in that sense, mentalists. We care about answering the “three big questions” about humans and language (Chomsky, 1986):

1. what a language user knows;
2. how that knowledge is put into use, to communicate, among other things; and
3. how that ability gets acquired.

Calling ourselves mentalists means that we think the right way to answer those questions is by “making reference to the mental events, capacities, and processes” of the language user (Katz, 1964). Needless to say, all those mechanisms are embedded in brains, which some of us study directly. We take, from the ‘generative’ in generative grammar, a commitment to being explicit in our formal models. We value careful attention to the description of specific linguistic systems and an ecumenical but rigorous approach to methods. We like to conjecture, to kvetch, and even to hear from people who think we’re wrong.

Why this blog?

It is an exciting time to be a linguist. The field has made remarkable progress on (1)–(3), and we think more of that progress goes unappreciated than it should. We want to celebrate it, and to make more room for the synthesis it allows. But while parts of the map are now filled in at high resolution, much terra incognita remains. So the time is ripe to reconnoiter, and to plan new expeditions.

It is also a vertiginous time to be a researcher in these areas. Scientific questions about language are coming into closer contact with neighboring academic fields and with technological advances, at what seems to be a quickly accelerating pace. We are enthusiastic about many aspects of these convergences and what we might be able to learn from them, but sometimes feel that important issues raised by the core questions in (1)–(3) above, which remain our focus, can get lost in the excitement.

That’s all about the moment we find ourselves in.

But then there’s the way we talk to one another. Traditional academic publishing doesn’t let us move as nimbly, range as widely, or speak as authentically as we want to. At the same time, we also wanted something more measured, and less evanescent, than microblogging on social media. We wanted writing that we controlled, published on our schedule, free to read and at whatever length the idea needs – and that others can respond to directly.

Finally, all of us were strongly influenced by Norbert Hornstein’s Faculty of Language blog – and we know we’re not the only ones! – so we wanted to create a space inspired by that, where we could spend some more time together.

How we want to proceed

Like many other blogs, this is a team effort. We don’t necessarily mean to speak in unison, but we hope there is some harmony. We discuss what topics and areas we’d like to say something about and we give each other feedback before publishing. Every post is written by its human authors.

Our posts are not all necessarily about “Big Questions”: we’re interested in many approaches, including targeted reactions, bite-sized tutorials, round-ups of recent research or contemporary reviews (in the spirit of what Glot International used to run as “State-of-the-Article” pieces).

We’re aiming to publish a piece roughly weekly. Eventually we’re also interested in publishing others’ pieces, and we’d love to hear proposals for topics from a wide variety of viewpoints. See the Contributing section of our About page for how to get in touch.

Finally but importantly

Anyone with a GitHub account can comment on posts. See How to comment for a quick guide.