When I began reading the chapter on representing meaning I was pretty sure I would find it intolerable. I was wrong however. It wasn’t a bad read even for someone like me who hates amodal representations and logical computations. This is not to say I agreed with it as a model for mental representations, but I did find it comprehensible at least.
I am familiar with these kinds of representations from my readings of Fodor, Pylyshyn, and Carruthers. They are hallmarks of classical cognitivism and the result of the invention of the universal Turing machine. They seem necessary to explain cognition but (fortunately) they are not. There are now dynamical system methods that can provide models without representations or computations. The problem with both models is that they can describe and predict behavior, but that is pretty much it. They are a form of dry instrumentalism, which many, including myself, find unattractive on their own without a theory behind them. Unfortunately, the theories that are provided to back them up are just theories which are often modified ad hoc to fit new data.
So this brings the obvious coherence question: Do you believe that this model of representation –really- what is happening in the mind based upon our knowledge of biology? If there were a model that fit the data just as well and that cohered with a more ecological framework of the mind (and there is) would you be willing to give up on these representations and switch to something new yet a little more “fuzzy”? Could anything possibly be fuzzier than this anyway?
I found the logical representations paper not only interesting, but especially helpful - I only wish I had read it two weeks ago. I read a paper that included statements written in logical form and I was clueless; like finding some tool in Level 2 of Zelda that you need in Level 4. I am now better equipped to progress.
(Tangential question: Are there "translators", programs that will code utterances/statements into logical form for a user?)
If an accurate knowledge base were established in the English language, where all utterances/statements could be accounted for with logical form, would this benefit translations of other languages? (Not sure how else to word that question.) Would it make for better teaching methodologies if the frameworks of each language could be isolated in this fashion?
I enjoyed the discourse on statives, activities, accomplishments and achievements. They did well to highlight the inherent differences in a way that a laymen such as myself could understand rather easily. I appreciate this sort of breakdown, where key terms are highlighted and expounded on in a useful fashion. I loved this reading. Truthfully I didn't want to stop at page 39, but other obligations arose... per the norm.
Is there an interface available to us that will allow for a more tangible understanding of the information here? A program that we can see use this input?
I enjoyed these readings quite a bit! I've always found J&M very readable though, particularly for computer scientists. Ben, would it help if I said I thought that they weren't proposing a mental model of anything, but rather they seem to be describing how to handle knowledge representation in a system. I think this is a chapter from a textbook, so it's more concerned with how to handle information and the state of the art, and not really modeling anything mental. If only I were that logical and linear thinking!
I really enjoyed Vasile's paper, and I'd like to see him explain more about it. I do hold some reservations about it functioning well in the domain of Biology. Just from our own experience with LSA and biology, we've found that it often confuses contrastive terms, like eukaryotic vs prokaryotic. They are so often mentioned in each other's company, they seem to be taken as being highly related, but when discussed, they are actually meant to be sort of "opposites". They can also broadly be described in similar ways, while their differences lie in the particulars. Biology seems to be fraught with things like that. Perhaps, though, there's enough details included in this method to set them apart.
When reading about knowledge representation in these two articles, I was trying to think of other elements of the text that would add to the meaning of the text. Specifically, the meaning that the author intended to convey to the reader. There can always be additions using something like LIWC , but I started thinking about book series that I read and how I kind of get to know the author's point of view and personality from reading (or at least what I perceive through their writing). This also made me think about how when you become more familiar with people, you can start to use a kind of conversational short hand because you know enough about them to know what they probably mean about certain topics.
I was wondering if there would be a way to build into these models some type of element of the author's or speaker's personality or writing style. I think that this could really add to the meaning in a way that many natural language processing techniques would miss. For example, these measures could encompass general tendencies or how frequently the author/speaker uses sarcasm. I realize that asking all authors/speakers to complete a personality questionnaire is not realistic, but I wonder if there is a way that this could be inferred from their writing? I just feel like whether it is an article in the newspaper, a short story, a series of novels, or even a blog post, having some type of understanding of the author's point of view/personality would add to the construction of knowledge representation.
I too enjoyed these papers. Not that I'm still not totally enamored with LSA, but NL-KR seems to address some of the issues with LSA that have been discussed previously (context, negation, syncopy, etc.). Blair, you beat me to what I was going to say! I was thinking about Roger Kreuz's talk about sarcasm and irony. These things along with metaphor and others, really make language "come alive" to us as perceivers. So while I think that these models demonstrate a great deal, there is still something missing from our 'real' natural language. One day, perhaps there will be a way for the program to first assess the overall theme of the document and account for things like tone, sarcasm, somberness, or what have you. And if it can do that, then perhaps negation and direction can also be addressed. I'm not a programmer, so don't ask me how that will work! And just to briefly add: Where have logical connective symbols been all my life? I'm not sure how I've never encountered them before the past couple of weeks. They seem to help bridge the gap (somewhat) between the natural and computational.
On page 3 of Jurafsky and Martin (2006), it talked about literal meaning of sentence. Is metaphoric meaning presented in this way? Meanwhile, it said it was separated from the context. I have a little doubt about this, for the context plays a pivotal role in understanding speaker’s or writer’s meaning. With the aid of context, much information will be lost or distorted. Thus, I am interested in reading the following chapters to see how to deal with metaphor and context.
On page 20, Fig 16.4, Truth table seems difficult to understand. Does P represent Proposition, and Q for quantifiers? And how to interpret this table? In the example: All vegetarian restaurants serve vegetarian food. The author uses universal quantifier to explain the representation of this sentence, but not use the existential quantifier. Is this for the word “all” in the sentence?
These readings were some of my favorites so far. I really liked the keywords being highlighted. It seems very menial, but I thought it was very helpful!
I found that distinction between vagueness and ambiguity to be interesting. It makes sense as it is explained here. I can't remember exactly if this was addressed in any of the LSA papers, but I am wondering how LSA deals with these two (vagueness and ambiguity) and how/if it relates to this paper? One more thing, it is mentioned that an underlying assumption is based on the fact that if you understand something, you can talk about it. I think there are times that I cannot explain things that I would consider myself to understand. Also, my understanding from day to day or instance to instance may vary ?
I was impressed with (and also overwhelmed by) the fact that the terminology I learned from my semantics class are now redefined by computer scientists. Even the logic behind are largely shared by the two, the nuisances involved across two fields of study can still be distinctively different. So, in general, I found the readings helpful but challenging. The readings are helpful in the way that it laid out how human language are represented in a set of logical theorem: By decomposing elements representing aspects of meaning (e.g. aspect/tense of verb), tearing apart the conceptual form from surface word form (and thus disambiguate meanings), quantifying variables in logical formulas, and judging the truth value of a statement by referring to its extension. Some questions: How to represent concepts that are gradient or relative (e.g. big/small, tall/short, blue/green/turquois)? Say,you don’t know that an elephant is big until you try to compare it with a mouse (or yourself). Also, does the article ever define what first order logic is?
In the article for the week, it was mentioned that there was a tradeoff between depth and robustness and ideally we would like be able to have both something robust and deep. Now if the goal here is to model human processes, then this made me wonder that perhaps we don't need a model that is always robust and always deep. In many instances when we process language it isn't deeply or robustly (I suppose in the case of being unfamiliar with certain words). Sure, for Watson this is great, but perhaps we don't actually do this and instead process information the same way as any of these models but contingent upon by the task at hand. In Vasile's example, it makes sense that a bag of words might be sufficient for super shallow processing, whereas it is insufficient when we are trying to thoroughly understand something. Same for framenet, sometimes so much information is necessary, but we might be lacking in our vocabulary. Are there any models that dynamically adjust on this spectrum of depth and robustness based on task goals? Or are most attempts aimed at making something both scalable and robust?
The Rus article reminded me about a debate we had in one of my classes in undergrad. We were reading a book called On Intelligence by Jeff Hawkins and Sandra Blakeslee. A point that was brought up in the book was the opinion that in order to create true intelligence in a machine, we should look at humans. In other words, look at their cognitive processes, and also at the physiological components (i.e., the brain). In my class we then discussed if that was the best course of action, or if it wasn’t necessary to understand and then replicate human processes. Just because we can do/have it, doesn’t mean there isn’t a more efficient way. Of course another huge issue is how to define “intelligence.” The beginning of this article just reminded me about that a lot… I’m not really sure why. Sorry I guess that was kind of a random thought.
Another completely off topic notion in case you want to get your brain juices flowing: the article cites part of the Declaration of Independence, “all People are created equal, that they are endowed, by their Creator, with certain unalienable Rights, that among these are Life, Liberty, and the Pursuit of Happiness.” Do you think that we are born with these rights? Who says we all have rights? Are they something we’re ‘born with,’ or a social construction? I think it’s interesting.
So finally to the Rus article itself. I’m unclear as to what the final product is. Is this LS-LF for the purposes of machine learning, or a theory of how people represent meaning? They talk about how human-friendly LS-LF is, so does that mean it is used by humans, rather than machines? Also, they lost me at the beginning of page 7 with the “energy:n:LSA-V4 (x2) & to:p:LSA-V5 (x2, x3).” I was pleasantly surprised by the other reading. I loved it. Not only was it clearly written and easily understood, but I like logic. Just the other day, before I started reading these articles, I needed a break from staring at the computer screen, so I was trying to put the phrase “Distance makes the heart grow fonder” into some sort of logical representation (although this sounds sarcastic, I’m actually being serious). Is there anyone else who enjoys that sort of thing, or is it basically just me?
Sorry that this post was all over the place and partially unrelated. This is just a glimpse into the way my mind functions.
I think one of the cool things about this logic approach is it attempts to represent meaning from full utterances and sentences. This seems very different from the other models we have covered so far that rely on the bag-of-words approach. This isn't to say that the BOW models have less merit (our readings have shown us that they are very useful), but I thought it was refreshing to learn about this kind of approach after getting BOW week after week. This also made me think back to the IBM super computer "Watson". It wouldn't surprise me if Watson used a combination of all the approaches we've talked about (plus many more) in its language comprehension
As others have discussed, the issue of representing nonliteral statements is an interesting one. I wonder if this can be tied into the "world-creating ability" of words and expressions that was discussed in the Beliefs section. If one can form logical statements that represent a hypothetical reality that isn't intended to be true in the real world, can the same be done for figurative meaning? I think it would be very difficult, but it's just a thought.
This week's reading is gotta be the most challenging so far, since it is almost purely computer science driven, I only understood on the surface level of the concerns and methods of how meaning is represented. The Rus paper is especially challenging.
What is most interesting to me has been the work on word sense disambiguation. Although I understand the usefulness of representing meaning using bag-of-words and LSA, and many of the disambiguation happens using heavy context. I can't help to think if this can be graphed onto how our brains work. Do our brains also store these meanings after using these ambiguous words in various context and then, having an enhanced reaction time to recognize and disambiguate words, or, if we are in contact with a brand new ambiguous word, we can either: a. decide with high inter-person reliability what we perceived the word to be, or b. immediately would require context in order to store and understand the new word's meaning?
When I began reading the chapter on representing meaning I was pretty sure I would find it intolerable. I was wrong however. It wasn’t a bad read even for someone like me who hates amodal representations and logical computations. This is not to say I agreed with it as a model for mental representations, but I did find it comprehensible at least.
ReplyDeleteI am familiar with these kinds of representations from my readings of Fodor, Pylyshyn, and Carruthers. They are hallmarks of classical cognitivism and the result of the invention of the universal Turing machine. They seem necessary to explain cognition but (fortunately) they are not. There are now dynamical system methods that can provide models without representations or computations. The problem with both models is that they can describe and predict behavior, but that is pretty much it. They are a form of dry instrumentalism, which many, including myself, find unattractive on their own without a theory behind them. Unfortunately, the theories that are provided to back them up are just theories which are often modified ad hoc to fit new data.
So this brings the obvious coherence question: Do you believe that this model of representation –really- what is happening in the mind based upon our knowledge of biology? If there were a model that fit the data just as well and that cohered with a more ecological framework of the mind (and there is) would you be willing to give up on these representations and switch to something new yet a little more “fuzzy”? Could anything possibly be fuzzier than this anyway?
I found the logical representations paper not only interesting, but especially helpful - I only wish I had read it two weeks ago. I read a paper that included statements written in logical form and I was clueless; like finding some tool in Level 2 of Zelda that you need in Level 4. I am now better equipped to progress.
ReplyDelete(Tangential question: Are there "translators", programs that will code utterances/statements into logical form for a user?)
If an accurate knowledge base were established in the English language, where all utterances/statements could be accounted for with logical form, would this benefit translations of other languages? (Not sure how else to word that question.) Would it make for better teaching methodologies if the frameworks of each language could be isolated in this fashion?
I enjoyed the discourse on statives, activities, accomplishments and achievements. They did well to highlight the inherent differences in a way that a laymen such as myself could understand rather easily. I appreciate this sort of breakdown, where key terms are highlighted and expounded on in a useful fashion. I loved this reading. Truthfully I didn't want to stop at page 39, but other obligations arose... per the norm.
Is there an interface available to us that will allow for a more tangible understanding of the information here? A program that we can see use this input?
I enjoyed these readings quite a bit! I've always found J&M very readable though, particularly for computer scientists. Ben, would it help if I said I thought that they weren't proposing a mental model of anything, but rather they seem to be describing how to handle knowledge representation in a system. I think this is a chapter from a textbook, so it's more concerned with how to handle information and the state of the art, and not really modeling anything mental. If only I were that logical and linear thinking!
ReplyDeleteI really enjoyed Vasile's paper, and I'd like to see him explain more about it. I do hold some reservations about it functioning well in the domain of Biology. Just from our own experience with LSA and biology, we've found that it often confuses contrastive terms, like eukaryotic vs prokaryotic. They are so often mentioned in each other's company, they seem to be taken as being highly related, but when discussed, they are actually meant to be sort of "opposites". They can also broadly be described in similar ways, while their differences lie in the particulars. Biology seems to be fraught with things like that. Perhaps, though, there's enough details included in this method to set them apart.
When reading about knowledge representation in these two articles, I was trying to think of other elements of the text that would add to the meaning of the text. Specifically, the meaning that the author intended to convey to the reader. There can always be additions using something like LIWC , but I started thinking about book series that I read and how I kind of get to know the author's point of view and personality from reading (or at least what I perceive through their writing). This also made me think about how when you become more familiar with people, you can start to use a kind of conversational short hand because you know enough about them to know what they probably mean about certain topics.
ReplyDeleteI was wondering if there would be a way to build into these models some type of element of the author's or speaker's personality or writing style. I think that this could really add to the meaning in a way that many natural language processing techniques would miss. For example, these measures could encompass general tendencies or how frequently the author/speaker uses sarcasm. I realize that asking all authors/speakers to complete a personality questionnaire is not realistic, but I wonder if there is a way that this could be inferred from their writing? I just feel like whether it is an article in the newspaper, a short story, a series of novels, or even a blog post, having some type of understanding of the author's point of view/personality would add to the construction of knowledge representation.
-Blair
I too enjoyed these papers. Not that I'm still not totally enamored with LSA, but NL-KR seems to address some of the issues with LSA that have been discussed previously (context, negation, syncopy, etc.). Blair, you beat me to what I was going to say! I was thinking about Roger Kreuz's talk about sarcasm and irony. These things along with metaphor and others, really make language "come alive" to us as perceivers. So while I think that these models demonstrate a great deal, there is still something missing from our 'real' natural language. One day, perhaps there will be a way for the program to first assess the overall theme of the document and account for things like tone, sarcasm, somberness, or what have you. And if it can do that, then perhaps negation and direction can also be addressed. I'm not a programmer, so don't ask me how that will work! And just to briefly add: Where have logical connective symbols been all my life? I'm not sure how I've never encountered them before the past couple of weeks. They seem to help bridge the gap (somewhat) between the natural and computational.
ReplyDeleteOn page 3 of Jurafsky and Martin (2006), it talked about literal meaning of sentence. Is metaphoric meaning presented in this way? Meanwhile, it said it was separated from the context. I have a little doubt about this, for the context plays a pivotal role in understanding speaker’s or writer’s meaning. With the aid of context, much information will be lost or distorted.
ReplyDeleteThus, I am interested in reading the following chapters to see how to deal with metaphor and context.
On page 20, Fig 16.4, Truth table seems difficult to understand. Does P represent Proposition, and Q for quantifiers? And how to interpret this table?
In the example:
All vegetarian restaurants serve vegetarian food.
The author uses universal quantifier to explain the representation of this sentence, but not use the existential quantifier. Is this for the word “all” in the sentence?
These readings were some of my favorites so far. I really liked the keywords being highlighted. It seems very menial, but I thought it was very helpful!
ReplyDeleteI found that distinction between vagueness and ambiguity to be interesting. It makes sense as it is explained here. I can't remember exactly if this was addressed in any of the LSA papers, but I am wondering how LSA deals with these two (vagueness and ambiguity) and how/if it relates to this paper?
One more thing, it is mentioned that an underlying assumption is based on the fact that if you understand something, you can talk about it. I think there are times that I cannot explain things that I would consider myself to understand. Also, my understanding from day to day or instance to instance may vary ?
This comment has been removed by the author.
ReplyDeleteI was impressed with (and also overwhelmed by) the fact that the terminology I learned from my semantics class are now redefined by computer scientists. Even the logic behind are largely shared by the two, the nuisances involved across two fields of study can still be distinctively different. So, in general, I found the readings helpful but challenging.
ReplyDeleteThe readings are helpful in the way that it laid out how human language are represented in a set of logical theorem: By decomposing elements representing aspects of meaning (e.g. aspect/tense of verb), tearing apart the conceptual form from surface word form (and thus disambiguate meanings), quantifying variables in logical formulas, and judging the truth value of a statement by referring to its extension.
Some questions: How to represent concepts that are gradient or relative (e.g. big/small, tall/short, blue/green/turquois)? Say,you don’t know that an elephant is big until you try to compare it with a mouse (or yourself). Also, does the article ever define what first order logic is?
In the article for the week, it was mentioned that there was a tradeoff between depth and robustness and ideally we would like be able to have both something robust and deep. Now if the goal here is to model human processes, then this made me wonder that perhaps we don't need a model that is always robust and always deep. In many instances when we process language it isn't deeply or robustly (I suppose in the case of being unfamiliar with certain words). Sure, for Watson this is great, but perhaps we don't actually do this and instead process information the same way as any of these models but contingent upon by the task at hand. In Vasile's example, it makes sense that a bag of words might be sufficient for super shallow processing, whereas it is insufficient when we are trying to thoroughly understand something. Same for framenet, sometimes so much information is necessary, but we might be lacking in our vocabulary. Are there any models that dynamically adjust on this spectrum of depth and robustness based on task goals? Or are most attempts aimed at making something both scalable and robust?
ReplyDeleteThe Rus article reminded me about a debate we had in one of my classes in undergrad. We were reading a book called On Intelligence by Jeff Hawkins and Sandra Blakeslee. A point that was brought up in the book was the opinion that in order to create true intelligence in a machine, we should look at humans. In other words, look at their cognitive processes, and also at the physiological components (i.e., the brain). In my class we then discussed if that was the best course of action, or if it wasn’t necessary to understand and then replicate human processes. Just because we can do/have it, doesn’t mean there isn’t a more efficient way. Of course another huge issue is how to define “intelligence.” The beginning of this article just reminded me about that a lot… I’m not really sure why. Sorry I guess that was kind of a random thought.
ReplyDeleteAnother completely off topic notion in case you want to get your brain juices flowing: the article cites part of the Declaration of Independence, “all People are created equal, that they are endowed, by their Creator, with certain unalienable Rights, that among these are Life, Liberty, and the Pursuit of Happiness.” Do you think that we are born with these rights? Who says we all have rights? Are they something we’re ‘born with,’ or a social construction? I think it’s interesting.
So finally to the Rus article itself. I’m unclear as to what the final product is. Is this LS-LF for the purposes of machine learning, or a theory of how people represent meaning? They talk about how human-friendly LS-LF is, so does that mean it is used by humans, rather than machines?
Also, they lost me at the beginning of page 7 with the “energy:n:LSA-V4 (x2) & to:p:LSA-V5 (x2, x3).”
I was pleasantly surprised by the other reading. I loved it. Not only was it clearly written and easily understood, but I like logic. Just the other day, before I started reading these articles, I needed a break from staring at the computer screen, so I was trying to put the phrase “Distance makes the heart grow fonder” into some sort of logical representation (although this sounds sarcastic, I’m actually being serious). Is there anyone else who enjoys that sort of thing, or is it basically just me?
Sorry that this post was all over the place and partially unrelated. This is just a glimpse into the way my mind functions.
I think one of the cool things about this logic approach is it attempts to represent meaning from full utterances and sentences. This seems very different from the other models we have covered so far that rely on the bag-of-words approach. This isn't to say that the BOW models have less merit (our readings have shown us that they are very useful), but I thought it was refreshing to learn about this kind of approach after getting BOW week after week. This also made me think back to the IBM super computer "Watson". It wouldn't surprise me if Watson used a combination of all the approaches we've talked about (plus many more) in its language comprehension
ReplyDeleteAs others have discussed, the issue of representing nonliteral statements is an interesting one. I wonder if this can be tied into the "world-creating ability" of words and expressions that was discussed in the Beliefs section. If one can form logical statements that represent a hypothetical reality that isn't intended to be true in the real world, can the same be done for figurative meaning? I think it would be very difficult, but it's just a thought.
This week's reading is gotta be the most challenging so far, since it is almost purely computer science driven, I only understood on the surface level of the concerns and methods of how meaning is represented. The Rus paper is especially challenging.
ReplyDeleteWhat is most interesting to me has been the work on word sense disambiguation. Although I understand the usefulness of representing meaning using bag-of-words and LSA, and many of the disambiguation happens using heavy context. I can't help to think if this can be graphed onto how our brains work. Do our brains also store these meanings after using these ambiguous words in various context and then, having an enhanced reaction time to recognize and disambiguate words, or, if we are in contact with a brand new ambiguous word, we can either: a. decide with high inter-person reliability what we perceived the word to be, or b. immediately would require context in order to store and understand the new word's meaning?