This article did well to highlight much of what seems to hinder natural language processing. It does seem possible, but I would argue most things are.
Machine learning I find to be one of the better methods to obtain this end. Also, the note about exploiting CYC more was well-deserved. Granted money has been "blown" on the project, it may still serve a purpose beyond it's original intentions.
Regarding machine learning: can a computer learn to use an alphabet? I know there are issues with comprehending punctuation, but is it possible to train a model from the ground up? The way you would train a child? Teach it the letters, then how they form words, sentences, and text? I have no clue how this could happen, but just wondering if it has been tried?
It seems like one of the major hindrances to NLP is that those seeking to initiate it don't have a good idea of how it works in a human's mind. Is it possible to ask a computer what it needs? I ask these questions sincerely. We utilize dictionaries, thesauri, encyclopedias, word of mouth, etc. to understand text. Can we train a computer to do the same? For it to conduct it's own research to produce understanding, or at least a proper inference?
I think the problem of dealing with generalizations (e.g. dogs have tails) is an interesting one, because it seems directly tied to the issue of fuzziness that we discussed in regard to concepts. We learned that the classical (definition-like) view of concepts is inadequate because of issues like fuzziness, exceptions to the rule, typicality, etc.
To me, this means that trying to represent meaning through logic when dealing with such concepts is especially difficult. For this to work in the case of machine learning, it seems like we have to incorporate extensive background knowledge for computers to pick up on exceptions, or use default logic that was discussed in the article. Just based on what I read in that section (and I may not be understanding this very well), default logic does not seem like a very elegant solution either. Granted, this seems like a problem that cannot be solved in a very tidy way.
This article really reminded me of how out of practice I am in logic. I enjoyed the parts I could translate, but most of it was really difficult for me. Is the point of computational semantics ultimately to create a computer that can think like a person?
My problem with most logic driven models of human thought are that they are so static and inflexible. I don't think that the mind works with logic-like computations on language-like representations. In my opinion there is a reason for this which is why these models are encountering such difficulty. It seems like the much of meaning is derived from context. If there was a way a computer could learn to read into context I suspect it would work better.
Maybe Luno has a good idea in that we need to build a mind from the ground up. Give it goals and time-pressure and the ability to learn/associate information it needs to achieve its goals. If it had an alphabet and a need to communicate in words and then sentences along with some kind of flexible way of learning then eventually it would possibly be viable. Maybe use one of those evolutionary simulations where only the best models survive to be used as the prototypes for the next generation of models until eventually a model that works well arises.
I don't know how machine learning research takes place though so hopefully no one takes this too seriously. I wouldn't mind a refresher on translations for the symbols and the logical propositions from the text.
While reading this article, I kept coming back to an issue I have with the process of any of these models of computational semantics. This issue is all these kind of models seem to be more additive than multiplicative. By this I mean that there doesn't seem to be a mechanism of accounting for the interaction of words. This is summed up in the sentence found on p. 345, "This strategy assumes that the meaning of a compound expression in computed from the meanings of its parts." So that's the additive issue that I have. We can construct better logistic models until we're blue in the face, but if it's only adding linearly, then we're missing a good portion of meaningful language. I thought the paper would address this when background knowledge was discussed. I was happy when lexical, world, and situational knowledge were acknowledged. This begins to address the concerns of myself, and many of you, about the importance of context. However, I still wish there was a way to incorporate how words or concepts can interact and become a new concept, or even change meaning (e.g., in sarcasm, irony, etc.).
I typed something in addition, but thought it was dumb. Mayhap I will formulate it better and bring it up in person. :)
The papers in these weeks are involved in first-order logic in NLP, which is pivotal for understanding the papers. Although some partial concepts are input in mind, it is better to know whether we have some basic courses systematically introducing this knowledge. Is this all covered in the course “NLP”? Or is there some other basic course? As for programming, which course is the basic to start with?
I thought I understood the meaning of natural language, but when I read this article, I am a little confused: What is the difference between human language and natural language? Wiki said: In the philosophy of language, a natural language (or ordinary language) is any language which arises in an unpremeditated fashion as the result of the innate facility for language possessed by the human intellect. Does that mean the human language is translated into the logic language, called natural language? Eg: Barbazoo found an egg. (This is human language)
Is the following first-order language the natural language?
Another question is about Fig 1: Seven entities in the domain is adopted by all the situation? How do we assume F (egg)={d2, d3, d4, d5, d6}? Why not only d2, since the sentence said “an egg”?
When this paper discussed how to handle metaphors I had a question. Given that people are aware that non-literal language and occurs during writing, how are they trying to handle this beyond simply using something like WordNet that might have both literal and non-literal meanings of the word?
Are there other lexical indicators that could allow for an algorithm to know when a non-literal sense of a word is being used? Or to search for an entire phrase (like for a frequently used metaphor)?
Also, when the paper began discussing constructing a source of background knowledge, I thought that through the interaction of background knowledge and context (or domain) the issue of non-literal language and ambiguities could potentially be resolved.
Possibly the problem of non-literal language could be handle by building in a prior knowledge of frequently used metaphors and analogies or that given a certain context, it is unlikely that a word would be used in a literal or non-literal sense. Perhaps it is the interaction of raw background knowledge and context specificity that will help to resolve non-literal language uses and other ambiguities when constructing a knowledge representation. So I guess another question that I have is, how useful is a large-scale, domain-free background knowledge? Can this ever be really useful or would you always have to incorporate some context-dependent background knowledge as well?
I am going to reveal a bias of my own. I ... don't like first order logic/language. I like some of the ideas behind it and its basic sentiment (reducing the world to a set of encoded, simple statements does seem nice), but I kept making the classic "disgust face" while reading this article. We picked on CYC for the last two weeks because it's unreadable to humans natively, but I think first order logic symbols are just as bad to the untrained eye. This article frames a lot of natural language problems as "things we solve to encode language in first order language", such as structural and lexical ambiguity, but should that really be the goal state? Statistical approaches abandon first order logic and they do alright, don't they? FOL just seems so inflexible and limited, where language is anything but. I guess I'm asking this: do we move forward with first order logic, or do we relegate it to a "fragmented" supporting role? Do we strive to overcome its limitations, or do we do something different? In the field of meaning representation or NLP, is there a movement away from first order logic?
Or am I just biased because I need to take a class in FOL?
I am not familiar with logic....at all. A few things that would be helpful in my understanding.
Why can't first order logic have some type of default inference?
I think that graph in table three makes sense and is helpful, but I would like it in class if we could possibly see examples of the various categories represented in the graph?
Out of the various types of background knowledge that are mentioned: "lexical knowledge (the meaning of words), world knowledge (general facts about the world, such as in the CYC project (Lenat 1995)) and situational knowledge (facts that hold in the current situation, only relevant for systems embedded in practical applications such as robots)" Which of these do you think are the most important if any? It seems that in some of the other things we have discussed, one or two of could be more important, albeit harder to implement?
I'm somewhat confused that it seemed that the topics that this week's paper addressed, majority of them were presented by John Sowa last week.
Nevertheless, from what I've read, it seemed that the author is closely crunching "logic" with Natural language. I'm not convinced that they can be classified together. Maybe on a lower level of discourse, logic can be applied in resolving ambiguities that natural language bring to the table. But when I think about language, I understand language to have a even higher level than "syntacts" "semantics" and "context/knowledge." it also carries aesthetics, literacy, non-literacy, humor, etc, that I think different usage of logic would not be able to address.
I was especially disappointed at the section that talks about computing background knowledge. I was really interested in how they would think of the ways to resolve this current issue, which John Sowa last week brought up. But instead, the paper started to talk about turning verb phrases in to nouns..... Ur, I really don't know how this is relevant to background knowledge. All this can do is to control for potential new nouns that came from verbs that the existing corpus do not have. I don't know if this is an attempt to try to create its own source of background knowledge, or they have a different definition of what constitutes as background knowledge I read the sections three times, still don't get the connection. It seemed that they defined it only on the vocabulary level, if so, I would like to say that I think this is ridiculous. I have read many papers on background knowledge in language and discourse processing. I'm unaware of any paper that defines background knowledge as the same as "vocabulary fluency."
Sorry this is tardy. I wonder if rules are more like guidelines that suggest which side of a continuum you are likely to find most of the objects belonging to a group. Considering the section on generics, these statements could be quantified, i.e., instead of 'a dog has a tail' we could look at tail length. Whereas most dogs have tails x in long, there are some that have shorter tails (all the way to 0 in) and some that have longer tails. But since we don't sit around trying to memorize the tail lengths of dogs, it's easier to think of these generics as truth statements. However, we would still be able to say that in general a dalmatian has a tail that is longer than that of a daschund… but sometimes a dalmatian has a tail that is shorter than that of a daschund. I don't think it would make sense that we know this because we have first order statements for every individual dog we encounter because that would be as impractical as the example of 6000000 U2 fans. Instead, couldn't we represent this information along a continuum, and use information regarding where most of the items along this continuum are as a heuristic?
I actually kind of enjoyed this article. Whitney questioned whether there was movement away from first order logic in the fields of meaning representation or NLP. I'd think not because this paper was published in 2011 and many of his sources are from the last ten years or so. I kind of thought using default logic to help with the issues that arise with generic statements was a decent idea, although I know not all of you agreed. To me, the limitation that with every new entity you have to include constraints as to how they behave, was a huge issue, so I was surprised more time wasn't spent on it. I was also disappointed that the section on fragments of first order logic didn't help clarify anything, for me at least. I wanted to know more about working with these fragments, especially since they refer to their use as a 'fix' for a few limitations. Can anyone 'fragment' as they see fit? And what exactly does that even mean? Does it involve bending some rules of first order logic?
This article did well to highlight much of what seems to hinder natural language processing. It does seem possible, but I would argue most things are.
ReplyDeleteMachine learning I find to be one of the better methods to obtain this end. Also, the note about exploiting CYC more was well-deserved. Granted money has been "blown" on the project, it may still serve a purpose beyond it's original intentions.
Regarding machine learning: can a computer learn to use an alphabet? I know there are issues with comprehending punctuation, but is it possible to train a model from the ground up? The way you would train a child? Teach it the letters, then how they form words, sentences, and text? I have no clue how this could happen, but just wondering if it has been tried?
It seems like one of the major hindrances to NLP is that those seeking to initiate it don't have a good idea of how it works in a human's mind. Is it possible to ask a computer what it needs? I ask these questions sincerely. We utilize dictionaries, thesauri, encyclopedias, word of mouth, etc. to understand text. Can we train a computer to do the same? For it to conduct it's own research to produce understanding, or at least a proper inference?
I think the problem of dealing with generalizations (e.g. dogs have tails) is an interesting one, because it seems directly tied to the issue of fuzziness that we discussed in regard to concepts. We learned that the classical (definition-like) view of concepts is inadequate because of issues like fuzziness, exceptions to the rule, typicality, etc.
ReplyDeleteTo me, this means that trying to represent meaning through logic when dealing with such concepts is especially difficult. For this to work in the case of machine learning, it seems like we have to incorporate extensive background knowledge for computers to pick up on exceptions, or use default logic that was discussed in the article. Just based on what I read in that section (and I may not be understanding this very well), default logic does not seem like a very elegant solution either. Granted, this seems like a problem that cannot be solved in a very tidy way.
This article really reminded me of how out of practice I am in logic. I enjoyed the parts I could translate, but most of it was really difficult for me. Is the point of computational semantics ultimately to create a computer that can think like a person?
ReplyDeleteMy problem with most logic driven models of human thought are that they are so static and inflexible. I don't think that the mind works with logic-like computations on language-like representations. In my opinion there is a reason for this which is why these models are encountering such difficulty. It seems like the much of meaning is derived from context. If there was a way a computer could learn to read into context I suspect it would work better.
Maybe Luno has a good idea in that we need to build a mind from the ground up. Give it goals and time-pressure and the ability to learn/associate information it needs to achieve its goals. If it had an alphabet and a need to communicate in words and then sentences along with some kind of flexible way of learning then eventually it would possibly be viable. Maybe use one of those evolutionary simulations where only the best models survive to be used as the prototypes for the next generation of models until eventually a model that works well arises.
I don't know how machine learning research takes place though so hopefully no one takes this too seriously. I wouldn't mind a refresher on translations for the symbols and the logical propositions from the text.
While reading this article, I kept coming back to an issue I have with the process of any of these models of computational semantics. This issue is all these kind of models seem to be more additive than multiplicative. By this I mean that there doesn't seem to be a mechanism of accounting for the interaction of words. This is summed up in the sentence found on p. 345, "This strategy assumes that the meaning of a compound expression in computed from the meanings of its parts." So that's the additive issue that I have. We can construct better logistic models until we're blue in the face, but if it's only adding linearly, then we're missing a good portion of meaningful language. I thought the paper would address this when background knowledge was discussed. I was happy when lexical, world, and situational knowledge were acknowledged. This begins to address the concerns of myself, and many of you, about the importance of context. However, I still wish there was a way to incorporate how words or concepts can interact and become a new concept, or even change meaning (e.g., in sarcasm, irony, etc.).
ReplyDeleteI typed something in addition, but thought it was dumb. Mayhap I will formulate it better and bring it up in person. :)
The papers in these weeks are involved in first-order logic in NLP, which is pivotal for understanding the papers. Although some partial concepts are input in mind, it is better to know whether we have some basic courses systematically introducing this knowledge. Is this all covered in the course “NLP”? Or is there some other basic course? As for programming, which course is the basic to start with?
ReplyDeleteI thought I understood the meaning of natural language, but when I read this article, I am a little confused: What is the difference between human language and natural language? Wiki said: In the philosophy of language, a natural language (or ordinary language) is any language which arises in an unpremeditated fashion as the result of the innate facility for language possessed by the human intellect. Does that mean the human language is translated into the logic language, called natural language?
Eg:
Barbazoo found an egg. (This is human language)
Is the following first-order language the natural language?
Another question is about Fig 1:
Seven entities in the domain is adopted by all the situation?
How do we assume F (egg)={d2, d3, d4, d5, d6}? Why not only d2, since the sentence said “an egg”?
When this paper discussed how to handle metaphors I had a question. Given that people are aware that non-literal language and occurs during writing, how are they trying to handle this beyond simply using something like WordNet that might have both literal and non-literal meanings of the word?
ReplyDeleteAre there other lexical indicators that could allow for an algorithm to know when a non-literal sense of a word is being used? Or to search for an entire phrase (like for a frequently used metaphor)?
Also, when the paper began discussing constructing a source of background knowledge, I thought that through the interaction of background knowledge and context (or domain) the issue of non-literal language and ambiguities could potentially be resolved.
Possibly the problem of non-literal language could be handle by building in a prior knowledge of frequently used metaphors and analogies or that given a certain context, it is unlikely that a word would be used in a literal or non-literal sense. Perhaps it is the interaction of raw background knowledge and context specificity that will help to resolve non-literal language uses and other ambiguities when constructing a knowledge representation. So I guess another question that I have is, how useful is a large-scale, domain-free background knowledge? Can this ever be really useful or would you always have to incorporate some context-dependent background knowledge as well?
-Blair
I am going to reveal a bias of my own. I ... don't like first order logic/language. I like some of the ideas behind it and its basic sentiment (reducing the world to a set of encoded, simple statements does seem nice), but I kept making the classic "disgust face" while reading this article. We picked on CYC for the last two weeks because it's unreadable to humans natively, but I think first order logic symbols are just as bad to the untrained eye. This article frames a lot of natural language problems as "things we solve to encode language in first order language", such as structural and lexical ambiguity, but should that really be the goal state? Statistical approaches abandon first order logic and they do alright, don't they? FOL just seems so inflexible and limited, where language is anything but. I guess I'm asking this: do we move forward with first order logic, or do we relegate it to a "fragmented" supporting role? Do we strive to overcome its limitations, or do we do something different? In the field of meaning representation or NLP, is there a movement away from first order logic?
ReplyDeleteOr am I just biased because I need to take a class in FOL?
I am not familiar with logic....at all. A few things that would be helpful in my understanding.
ReplyDeleteWhy can't first order logic have some type of default inference?
I think that graph in table three makes sense and is helpful, but I would like it in class if we could possibly see examples of the various categories represented in the graph?
Out of the various types of background knowledge that are mentioned:
"lexical knowledge (the meaning of words), world knowledge (general facts about the world, such as in the CYC project (Lenat 1995)) and situational knowledge (facts that hold in the current situation, only relevant for systems embedded in practical applications such as robots)"
Which of these do you think are the most important if any? It seems that in some of the other things we have discussed, one or two of could be more important, albeit harder to implement?
I'm somewhat confused that it seemed that the topics that this week's paper addressed, majority of them were presented by John Sowa last week.
ReplyDeleteNevertheless, from what I've read, it seemed that the author is closely crunching "logic" with Natural language. I'm not convinced that they can be classified together. Maybe on a lower level of discourse, logic can be applied in resolving ambiguities that natural language bring to the table. But when I think about language, I understand language to have a even higher level than "syntacts" "semantics" and "context/knowledge." it also carries aesthetics, literacy, non-literacy, humor, etc, that I think different usage of logic would not be able to address.
I was especially disappointed at the section that talks about computing background knowledge. I was really interested in how they would think of the ways to resolve this current issue, which John Sowa last week brought up. But instead, the paper started to talk about turning verb phrases in to nouns..... Ur, I really don't know how this is relevant to background knowledge. All this can do is to control for potential new nouns that came from verbs that the existing corpus do not have. I don't know if this is an attempt to try to create its own source of background knowledge, or they have a different definition of what constitutes as background knowledge I read the sections three times, still don't get the connection. It seemed that they defined it only on the vocabulary level, if so, I would like to say that I think this is ridiculous. I have read many papers on background knowledge in language and discourse processing. I'm unaware of any paper that defines background knowledge as the same as "vocabulary fluency."
Sorry this is tardy. I wonder if rules are more like guidelines that suggest which side of a continuum you are likely to find most of the objects belonging to a group. Considering the section on generics, these statements could be quantified, i.e., instead of 'a dog has a tail' we could look at tail length. Whereas most dogs have tails x in long, there are some that have shorter tails (all the way to 0 in) and some that have longer tails. But since we don't sit around trying to memorize the tail lengths of dogs, it's easier to think of these generics as truth statements. However, we would still be able to say that in general a dalmatian has a tail that is longer than that of a daschund… but sometimes a dalmatian has a tail that is shorter than that of a daschund. I don't think it would make sense that we know this because we have first order statements for every individual dog we encounter because that would be as impractical as the example of 6000000 U2 fans. Instead, couldn't we represent this information along a continuum, and use information regarding where most of the items along this continuum are as a heuristic?
ReplyDeleteI actually kind of enjoyed this article. Whitney questioned whether there was movement away from first order logic in the fields of meaning representation or NLP. I'd think not because this paper was published in 2011 and many of his sources are from the last ten years or so.
ReplyDeleteI kind of thought using default logic to help with the issues that arise with generic statements was a decent idea, although I know not all of you agreed.
To me, the limitation that with every new entity you have to include constraints as to how they behave, was a huge issue, so I was surprised more time wasn't spent on it.
I was also disappointed that the section on fragments of first order logic didn't help clarify anything, for me at least. I wanted to know more about working with these fragments, especially since they refer to their use as a 'fix' for a few limitations. Can anyone 'fragment' as they see fit? And what exactly does that even mean? Does it involve bending some rules of first order logic?