Knowledge Engineering & NLP - NLP Modeling chat
IRC Chat Pages [ Back to Chat Index Page ]
Hi. folks! Welcome to this chat on Knowledge Engineering. Tonight I'm going to cover my perception of why Knowledge Engineering is a highly useful methodology for rapidly learning stronger modeling skills. Clarify your understanding of modeling & of why AI as a field is highly useful for us as NLPers!
OK, let's jump right in & get started! Anyone here have an American Express Card? No, I don't want to charge you (!) -- I'm asking for another reason! To those of you who do, have you ever tried to use your AMEX card somewhere unusual, and the authorization came back with a request to call in? I have! This happens a LOT with Amex, and increasingly so with other cards.
|Tangram||I haven't, but a friend's did. She was on vacation in London. They noticed that charges were being made in London, so they called her home, and deactivated her card.|
|Gunny||My mom's card did; when I ordered too much, they said "over the limit."|
|Sanuk||They bounced mine (VISA) for over a daily limit.|
|Jonathan||Limits are different than what I'm talking about. I'm talking about a perfectly good card where you were charging something considered "OUT OF PATTERN." Now, what the hell does that mean?|
|RonDeVue||Out of pattern?|
|KevinM36||This happened to me when my wife went on a trip, and we were both actively using an AT&T calling card.|
|CyberSage||Or you buy regularly at $100 and then all of a sudden, you buy something that costs $1000.|
|Jonathan||Yes to all of the above. It also means that they keep track of your charges and the TYPE of charges you make, and the sequencing of charges. Then, if you suddenly charge something highly out of character for your habits, they flag it for a phone call. This really happens! Try charging only business travel expenses for a straight year then using it in a hardware store.|
|Tangram||Yes! Another upscale friend was making small purchases at a Caldor's. We think he triggered the "this guy had his card stolen" pattern -- small purchases in a "lower class" store.|
|Gunny||Oh yeah, that also happened with me! My mother doesn't use it that much, then I put $1000 on it for my computer!|
|Floydd||It's to prevent theft purchases.|
|Jonathan||Yes! That could be annoying for folks who are typically always in great financial positions, could it not?|
|Gunny||Hey, it's for our own good.|
|RonDeVue||They set up a buying profile on you, so to speak.|
|Jonathan||Yes, Gunny & Ron! Gunny, it may protect us & the card companies from fraud, but it isn't necessarily for our own good. My opinion: it nearly eliminates whatever privacy of information you thought you had. OK, folks, let's keep moving: Have ANY of you called a computer tech support group and actually gotten RAPID resolution on a problem? I.e, the person you spoke to intuitively knew how to solve your complex problem? Asked ALL the right questions in a perfect sequence?|
|RonDeVue, Bridge, BHope Tangram, KevinM36||Yes.|
|Jonathan||And then MAYBE they said something which told you that even though they got the PERFECT solution, the *individual* you spoke with ALSO apparently had no idea where the on-switch was to his computer. The cases I'm describing are where businesses increasingly use Artificial Intelligence technology in real-world situations to make what are considered "Intelligent" decisions, consistently, and quickly. Now -- We've heard the term Artificial Intelligence but what does it mean to YOU? What have you heard?|
|Gunny||A computer being able to think like a human.|
|BHope||A machine capable of making decisions|
|Tenore||A less than human, flexible mimicking of a thought process...|
|Sanuk||AI is distinct from Expert Systems. Expert systems is a computer modeling human experts. AI is a computer learning for itself|
|Jonathan||Yes Sanuk, I did knowledge engineering, building expert systems, for 7 years. I will get to that from a different angle.|
|Ommadawn||I think of expert systems when I think of AI.|
|RonDeVue||I do too, I'm working on designing a little self-learning into a couple of the programs I'm writing at the moment.|
|Floydd||Depends on whose account of AI. Classical? Allen Turing's account? Searle?|
These are some good answers. "Artificial Intelligence" is the name applied to the pursuit of making "intelligent" machines/computers. And there are a LOT of different approaches to this. From the theory-ridden early days of AI, there emerged two primary approaches to AI. LEARNING SYSTEMS & DECISION SYSTEMS.
At the time, they had to take their model of human thinking and break it into pieces they could replicate with some level of accuracy (Only as accurate as their measurements were, of course). Now -- as to LEARNING systems what emerged were called NEURAL NETWORKS & later GENETIC ALGORITHMS. But the 1st real successes with AI -- FEASIBLE financially viable successes -- were with DECISION systems. Also known as EXPERT SYSTEMS. Now, here's where we BEGIN to START to draw parallels!
They're called Expert Systems because they are software systems designed to MAKE DECISIONS, just like human experts do/did/would do. Humans aren't machines, we don't ALWAYS do things the same way, but it is possible to write software to flexibly apply criteria against a solution in reproduceable sequences -- such that IF you can somehow ELICIT the KNOWLEDGE that an expert knows, then you can build the software to do the "right" thing in the right circumstances. Make sense? (y/n)
|Floydd, CyberSage, Gunny, Gaijen, Steve_L, Sanuk, Tenore, NLPmind, Ommadawn, Bridge, KevinM36, Tangram||Yes!|
Who sees a connection yet between what I'm touching on -- and NLP Modeling? (that's rhetorical!) The problem for them at the time was to come up with software SHELLS that would allow a person to RAPIDLY code someone's knowledge without extended programming tasks. That was solved over time and there are some VERY sophisticated expert systems shells available today, across a WIDE spectrum of costs. Those costs are usually roughly relative to the extent to which you can get moment-to-moment support -- or NONE at all.
Now. Once you have the software, there's another problem -- you need to train someone to get INTO someone's head, so to speak, and PULL the knowledge out of the expert. Sometimes by tooth & nail! Here's a major dilemma: Experts don't have a freaking CLUE *how* they know what they know -- they just KNOW it! They spent years chunking up and its just IN there. And Knowledge Engineers (thats their title) are people trained to get in there & get it out, encode it, and build expert systems software. They have to know what the clues are for whats redundant, whats deleted/omitted. They have to be able to construct cases/situations that cause the expert to access the knowledge that they DON'T KNOW they have! Make sense still?
|CyberSage||If you explain shells, then yes! :)|
Briefly, a shell is an EMPTY expert system environment designed to make the entry of knowledge -- SIMPLE & FAST.
Now, just to give a little context, those that know a little about me know that I haven't been training all that long; about 2 years. Which doesn't seem like a lot of time. Not even to me! Before getting headlong into NLP -- I was a senior Knowledge Engineer for the world's most successful Expert Systems company. And I'd been an author/presenter at the American Association of AI conference, -- for writing an Innovative Application of AI.
|Jonathan||How many of you have written personal checks at a CompUSA, or Computer City, or any other major retail center to pay for something, and they approved or declined your check? There is a weird form of 'insurance' that retail centers can get called "check guarantee services." There are several competing services, but one of the biggest ones is Equifax Payment Services. I wrote/delivered their expert system.|
|RonDeVue||So you are the one! LOL!|
|Jonathan||Hey, their business rules, not mine; I'm just the one who prevented you from getting stopped more often! I.e., I didn't make up the rules, I just made them work! You can thank me for being ABLE to write checks! *grin*|
|CyberSage||Actually, he's right; the system was worse before that.|
|Jonathan||Seriously! I wasn't on the project for that long -- it was about to be scrapped when they brought me in. 1 year of floundering -- then I showed up & in 6 months, I re-engineered their system and delivered it, up & running. Then we wrote a paper on it & got it published at AAAI/96.|
|RonDeVue||I had an issue with you making it possible for my wife to write checks... and write checks... and write more checks...|
|Jonathan||Now. That's not a big "brag" since ANY of the people I worked with could have done the same.|
Yes, could have. I give them that -- there were indeed people in my consulting org who could have done better or done it faster. No kidding. And also on the flip side, there's a huge difference between people who have solid Knowledge Engineering skills -- and those who never learned it well. Companies often try to build such systems on the cheap, from within. It almost always turns out to be a far far less expensive effort to pay top consulting rates to the people who have a consistent history of project delivery.
OK, let's back up. When you know how to elicit the kinds of decision making other people make -- (as I did back then, without any framework for NLP™ at the time) then you're good at doing something required for expert systems, called Knowledge Acquisition. Big term for something Very Simple! Modeling.
Now, this is not what in NLP™ is called Strategy Elicitation. Modeling is a mathematical logical process whereby you identify a calculus for a model of thinking or expertise. Such that the calculus or cohesive model includes all the major variables -- or qualities that change from one situation to another -- that adjust the outcomes. And then if that calculus is then applied in such a way as to predict outcomes effectively, or produce similar results again, consistently, then you've produced a model (of unknown quality, just from that description).
That is what must be done to engineer knowledge. Now. We're not yet onto learning systems; we're still just dealing with decision systems. But ask yourselves this question & let it sink in for a while: What if you could build into your brain a way of automatically optimizing the decision systems you build in your brain? Everyone with me? (y/n)
|Gunny, Sanuk, CyberSage, Mezmer, Pwrr, Gaijen, RonDeVue, BHope, KevinM36, Ommadawn||Yes.|
Moving right along, now, the major detriment to Expert Systems alone is that they represent a momentary snapshot of a big knowledge picture. They don't grow very well, given the software shells, and given the limitations of the ES model!
It is easily feasible to edit the knowledge... bit by bit... long term over a lot of time. Discretionary adjustments are usually fine. But speaking on the basis of financial viability, Knowledge Bases rarely survive major overhauls. They don't seem to operate well after several re-engineering phases; too many band-aids, perhaps. And often, after a few years, many knowledge bases do require such overhauls. Why? Because the companies embody & accumulate more business expertise & information, and their in-house experts & employees learn more. And they want their systems to grow with what they've learned.
|RonDeVue||The knowledges are static in comparasion|
|Jonathan||Yes! Knowledge in Expert Systems are essentially a big collection of free-floating, unsequenced if-then statements. That's a ridiculously simplified explanation, but for the time being, it can suffice. Logical inference presides in this model; semantic value is ignored or inferred. (just as we deal with the structure of experience in NLP™!).|
All factual data is applied to chains of rules... to produce results. Now chunk back up! Learning Systems are amazing because we CAN indeed teach computers new things. We have neural networks, genetic algorithms... and other techniques for learning, for actually absorbing new knowledge.
The Army once built a neural network -- a software program with a virtual "black-box" brain, so to speak... that gets "trained" to recognize tanks in a field. They fed this neural network a 'training set' of photographs showing tanks in fields. The Neural Network said "yes there's a tank in this one.... no there's no tank in this one." Over time the net was trained to accurately recognize tanks in fields.
Neural Networks can not logically tell you HOW they learn, or HOW they know they have the right answer. When the neural nets are being trained, they're given sets of training data along with the right answers. So they learn that this picture gets this answer, and that picture gets that answer. They train themselves to provide responses that match the training data's expected results. They *tune* themselves to get the 'right' answers. Its still a highly valuable technique because all you have to do is feed in the training set and GRADE the responses that come out (& yes, that's a simplification). Again, the problem there is there's no way to find out how it learned.
Here's why that's problematic: The Army, all spit-shined & hot-to-trot, took its fancy new system out into the field and had it look at real tanks rolling through the hillside. And EVERY SINGLE time, it screwed up & got the wrong answer, Nope, no tank there... This obviously did not please the Army.
So the generals asked "so WHY isn't it working?" And the techs said, "we don't have a clue." And they were telling the truth; no one thought to recognize that every photo they fed to the system was of a SUNNY day... until later -- after they'd evaluated the problems for a while. It rained on the day they tested the system in the field. Now, they were lucky. Not every neural network gets that far. But its still a useful approach. You know how to learn VERY VERY WELL.... don't you? In at least some context? And you know how to rationally make decisions in some circumstances, VERY VERY WELL.... right? So... if one approach (expert systems) is incredible for making straight-forward rational decisions AND can tell you EXACTLY how it made up its mind about something (and it can, they're very good for that!)
if another approach (e.g. neural networks) doesn't have a clue HOW it learns but its amazing at learning...
Where do you think we want to look when we're exploring advancing HOW we optimize our internal processes? Do we even look at that? We *could* abandon ALL that -- and I think that may be a useful choice in many contexts. And we can also look very closely... and do what I think you NLPers out there have heard before... take what's useful & Discard what's not! So what if we took the best of the learning systems... and the most effective parts of decision-systems... and you fuse them together inside your mind (take a moment to do that now)! Then for example you might get something like this: A hybrid system where you optimize your results -- by distributing the tasks!
|Jonathan||If anyone here is thinking "parts model," screw that!|
|Tranzpupy||Screw the "parts model!"|
|Jonathan||Trust the Neural Network to produce new rules on the fly... new rules that get integrated into the rigid knowledge base. Now. Think of your STRATEGIES.... for a moment, as being rules. Do you want wishy-washy strategies? Or once you get GOOD ones or BETTER ones -- don't you want them to operate consistently well? You want reproduceable consistent results or better right?|
|Gunny, KevinM36, Ommadawn, Mezmer, Liketrane, CyberSage, RonDeVue, Bridge, Steve_L, bhope, Sanuk||Yes!|
|Jonathan||Well, you don't want your rules or your strategies to apply or not apply in wishy-washy ways.... based on whether the sky is fuscia or purple... :)|
|Gunny||Yeah, the weather shouldn't matter.|
|Jonathan||You want them to operate consistently. And, you want the ability to flexibly create & try out new rules on the fly.|
|RonDeVue||Generalize enough to operate independent of context?|
|Jonathan||You want to be able to discard & try on & play with them & change them, and while you try them out -- you want them to be reliable. And, you may learn something while you run through a strategy or apply a rule that allows you to make adjustments. But. A hybrid integrated system may be an ideal way to make OPTIMAL use of these intelligent qualities... in such a way that explores the fringes of how effective you can be. Years ago -- 9 to be exact, a professor of mine built the 1st effective hybrid system for use in predicting stock results.|
|Jonathan||And his results flipped out the major investment houses while they scrambled to build their own.|
|Ommadawn||Did it have anything to do with Pi? (I love retrograde mercury!)|
|Gaijen||Yes I've heard they're very popular these days.|
|Jonathan||Here's the problem with financial applications of intelligent computing: With financial prediction, you have hundreds of rules which are applied to the market and trend analysis to determine a good stock pick. But guess what: The Rules Change Day to Day! Which means, any expert system you build -- is useful ONLY for the next day's predictions! Too many variables!|
|Gaijen||And that doesn't even take into account the whole system involved in what makes stocks go up and down.|
|Jonathan||Thus Expert Systems are not a good choice. And yet, investors need to be able to explain why something is a good pick. For that, where AI is concerned, they NEED an expert system's ability to explain decision-making! So what he did was -- build & train a neural network to look at a plethora of indicators from the last week's worth of market data, after which it produces a new knowledge base (a knowledge base is a collection of rules!). And in turn, that new knowledge base is then used to predict stock results... with ridiculous accuracy (95%, perhaps). *WHY* on Earth do you think we've had such a BULL MARKET?|
|Ommadawn||I find that to be just a little bit creepy|
|Jonathan||Ommadawn, I found it creepy, too, at first. These major investment houses (the ones where top financial advisors manage mutual funds) do far better on average than the private investor. Because they've sunk HUGE amounts of money into these kinds of systems. Now here's what's funny -- since all these systems do is "reproduce" intelligence: a TRULY savvy investor -- one who knows the market inside & out -- one who knows the companies he's considering investing in -- longterm and shortterm history -- THAT kind of investor may regularly do far better than these systems do.|
After that ONE investment house started dominating the market, it took other investment houses a few months to catch up. This caused what we might call a shortterm blip in the market. And that exemplifies AI evolving in the marketplace. But you don't have to look there. You can just look innnnnnnside. To find the most advanced REAL intelligence there is. Because just as we try to make computers think like people... we can try to make ourselves think like computers (in SELECT circumstances) and we can learn from the field which has done more for cognitive thought modeling than any other field around -- CERTAINLY more than psychology's done.
As NLPers... AND As Knowledge Engineers... WE LOOK AT HOW. Screw "why." So to step outside a bit -- I did that for businesses for a few years, and when I started getting turned on to NLP™ -- it was like *click*, *click* -- I get all this -- and in some cases I had a lot of nuance work to do -- and in some cases I just had to allow for new Names for things I was already doing well. That should explain a little of the rapidity with which I integrated NLP™ skills in that 1st year after hearing about it.
Fortunately I also learned that the more you learn, the more there is to learn, so the initial overconfidence was replaced by a more open mind and a willingness to keep exploring & pursuing mastery & having a great time getting results!
Now, in practice, when you explore some of the Knowledge Engineering skills -- for encoding knowledge... you learn how to elicit strategies effectively, and the more you do it, the closer you're getting to learning how to REALLY model behavior & knowledge. So -- in London -- I'm going to lead people through learning how to Acquire Knowledge through some KE techniques, encode that knowledge, and then apply optimizing filters to it to help get to the most concise & elegant representation of that. And then USE that encoded knowledge to handle new circumstances well. And to then find out how much more smoothly their recognition of NLP™ patterns will occur! Not to mention how much more easily they'll understand and USE complex syntax. Not to mention how much more easily their use of the MetaModel will flow.
|KevinM36||Will there be tapes?|
|Gaijen||Yeah will there be a tape set and manual?|
|Floydd||And a coach?!|
|Jonathan||Hmmmm.... could be! Now -- I DO have more -- and I want to open it up to questions!|
Q & A!
Let's finish with opening up the floor to questions regarding the AI & KE & NLP covered tonight!
Floydd's question 1st! Would you clarify your question for the channel: "how does that evaluate the classical AI (computer program) account of the intentionality (aboutness) of people's beliefs?"
|Floydd||That's my question... the philosophy of the mind that states that AI contains within it the strategy of intention, as in the intentionality of people's behavior attitudes and beliefs...|
|Jonathan||Hopefully I've painted a picture where "AI" as an entire concept was & still is so nominalized that there's nothing you can do with it at that level. So. How does your question relate to how "AI" as a classical approach needed to fractionate into learning systems & decision systems & even other forms of AI like 'Machine Learning' & 'Data Mining' in order to make any headway?|
|Floydd||Basically, it's an open question.|
|Jonathan||Right. And it was opened in the 60's and its still open. Arguably you could say it was opened long before that through mathematical & philosophical exploration, but in terms of potential practical AI development, it was first explored in the 60's.|
|Floydd||I see. So what is your personal take on it? There is no "wrongness" or "rightness" in open questions.|
|Jonathan||I agree! I want that to remain an open question! I think few if any answers will be of much use there, at that level; I could be wrong. But here is one possible answer, for those intent on hanging tightly to the theoretical: Intention and "aboutness" is encoded in decision systems automatically by virtue of what is determined to be the outcome or the goal -- of chaining rules in a certain direction. I.e. intentionality goes to context & desired outcome, and higher level purposes. And the process of engineering knowledge is the process of determining whats important in their decision-making sequences, and also ignoring what isn't important. I.e, you know where you want to go or what potential outcomes there are, and you utilize present information or inferential reasoning to determine whether you get one outcome or another.|
|Gunny||So basically you just ask them a bunch of questions?|
|Jonathan||Gunny, if it were that simple, everyone would be doing it and earning $100,000-plus/year as a Knowledge Engineer. Avoid the word "Just" -- it presupposes closedmindedness to other possibilities.|
|Tenore||How about sentence completion: One KE technique I can use right away is: ______________________.|
|Jonathan||Tenore, let's ask the group: "Did any of you learn any 1 thing you can use right away from this chat?"|
|Gunny||Effect decision-making & rule-learning.|
|Liketrane||Yes, combining my decision-making and learning systems will make a big difference! They depend on one another.|
|KevinM36||I'm hip on developing some hybrid systems in my head. Also, what I know of AI has been clarified to a very useful point.|
|Jonathan||I have an automatic continual feedback loop -- just as YOU do -- that I'm just a bit more aware of -- that tests the decision making against new information as it comes in. Now.|
|Liketrane||How do we use this approach to identify and help alter another's perceptual grids?|
|RonDeVue||Ahhh, the 'how'!|
|Jonathan||Ahhhhh, Liketrane.... for that, they either have to do it on their own or come to London or Melbourne! Though I gave it away there :) THAT is the metaphor driving the actual training. My intention is to build a more optimal process for doing that for/with the attendees, whether they're aware of how I'm doing it -- or NOT. I want to help build better & more FLEXIBLE expert system generators... in you... or -- in my attendees. So those who come, will learn consciously & unconsciously how to optimize their hybrid connections of how they're learning and they're deciding processes.|
Other Knowledge Engineering resources.
There is much more to learn about -- including what I'm not doing just here in a Chat, -AND- including exercises to actually do a bit of knowledge engineering, AND including a working innate & practical understanding of a few Logic & Theorem proving techniques, like... Modus Ponens... Modus Talens... & Resolution. At the same time --
This chat DOES cover from a birds eye view an enormous amount of the connections between the two systems... those who have some knowledge of both will find it highly enlightening... those who have some or more knowledge of one may find the other more useful to learn about. Lastly, I was asked to recommend some Courses & Books! For those who wish to read more, look over these books, listed in order of my preferences, if you're looking for good, easily accessible material on AI & Knowledge Engineering. These books range from ~ $40-90. (Some are textbooks but are still excellent!) Check out these titles through your favorite BIG computer bookstore, or, through Amazon.Com:
|CyberSage||What about Journals or Magazines; any good industry ones?|
|Jonathan||Well there we're in a quandary -- There used to be AI Expert mag, which went under, then they came back up with a 1-time special release that was REALLY good. I found most of the textbooks to be SORELY lacking in practical usage information. The REAL gold is to be found in the manuals being written by the expensive Knowledge Engineering companies to support their software! They're very expensive to get, without spending thousands per seat/computer.|
|Floydd||Jonathan.. my professor on this subject knows virtually nothing about what we're talking about; he just knows what he reads. One day in class, I raised my hand to refute his point (with Sleight of Mouth!). I totally baffled him, in response to which he basically spoke in circles until I decided not to waste anymore of the class' time.|
|Jonathan||Again, most of the other textbooks do NOT impress me, because a lot of academicians are stuck in their little research boxes, publishing papers, getting barely-solvent hungry grad students to do most of their coding for them, while they write proposals for grant money -- they're not out in the real world making it work, or finding out what doesn't work & then finding other ways, within viable real-world deadline environments.|
|CyberSage||Pragmatics is good... if it don't work... why bother? Well, Jon, I feel naive... and I LOVE IT... more to learn and grow... great to have someone with advanced knowledge on this topic so readily available. Thanks for doing this chat.|
|Jonathan||Sure! As for courses -- MOST of the courses at universities... that are explicitly titled "knowledge engineering" and they would probably be a good place to start, as long as you remain aware of the absence of much practical knowledge there -- about how AI is actually applied in industry today. Make sure they're teaching using a current Expert System shell. Prolog is great for simple theory but drop it fast when you get into more knowledge-intensive systems. Make sure they're using CLIPS, or even the old E-MYCIN shell.|
|CyberSage||What is a current ES shell?|
|Jonathan||There are many many shells available these days, Phoenix. The best & most efficient/fastest/most functional ones are the ones that cost $1000 or more per seat/copy.|
|Gunny||Are those programming languages?|
|Jonathan||They are examples of Expert System "shell" packages -- empty engines.|
|CyberSage||Cool, cool, cool, Phoenix has more toys to play with... wheeeeee!|
|Tranzpupy||I once helped code an expert system shell.|
|Jonathan||Cool, tranzpupy! As part of the project, did you build a pattern & join network? That is what I did in London, by the way, at the Knowledge Engineering & NLP workshop in May 1999. I built internal Pattern & Join nets in the mind!|
|CyberSage||Will you do one state-side? London is too far when I am babysitting a business. Come to San Jose! I would love to have this taught there. I could even get you the venue! You know, since O'Connor wrote on systems theory... I really could see you... NOW.. writing a book about this and a followup with NLP/DHE™ integrated.|
|Jonathan||Thanks for the suggestions, I'll think about it! I'm going to keep it to just those two venues for now -- London UK. I will tape at least one of them. But the tape won't include footage of people doing the exercises, so if you want the knowledge and the experience, come to the seminar! Thank you for coming & joining & listening tonight! This one was a bit less interactive but perhaps more CONTENT-driven than some previous chats! I'm glad you could make it, and I look forward to the next one -- and the next one -- and to seeing everyone sometime at a training!|
|Dluzi||Hey, thanks Jonathan, for the informative and interesting evening!|
|Jonathan||Dom & Phoenix! Thanks for joining!|
|CyberSage||You are very welcome. I think it was your best chat I've seen to date... Kudos and I look forward to your next talk on this and other interesting topics! :)|
|Tenore||Remember to ask "what you don't know yet," and let that guide your decisions, automatically! Thanks for the evening.|
|Jonathan||Thanks to Kevin & Kathleen & John!|
|Dluzi||Thanks again, Jonathan. Good night all! Keep the fire buring bright!|
IRC CHAT OVER!