From crose Wed Jan 29 18:38:42 1997 Return-Path: Received: by MOGLI.rutgers.edu (4.1/25-eef) id AA03718; Wed, 29 Jan 97 18:38:16 EST Date: Wed, 29 Jan 97 18:38:16 EST From: Christopher Rose Full-Name: Christopher Rose Message-Id: <9701292338.AA03718@MOGLI.rutgers.edu> To: 330_542 Subject: first assignment Status: RO HI Folks, SO far so good... no cancellation yet. So start and finish the first assignment (it's a probability review). I'd like to get latex solutions made up for the other problem sets as well. Might someone out there be interested in such a task? Cheers, Chris Rose From crose Mon Feb 10 21:59:56 1997 Return-Path: Received: by MOGLI.rutgers.edu (4.1/25-eef) id AA19582; Mon, 10 Feb 97 21:59:52 EST Date: Mon, 10 Feb 97 21:59:52 EST From: Christopher Rose Full-Name: Christopher Rose Message-Id: <9702110259.AA19582@MOGLI.rutgers.edu> To: 330_542 Subject: test Cc: crose@MOGLI.rutgers.edu Status: RO Hi Folks, This is a test. If you did not get ths mailin then you're not on the list. Please send info asap if you want to get in on all the discussion. To start things off, I'd like someone to give the answer to problem 2.13 (coin weighing) part a. Then I'd like for a discussion to erupt on whether the bound is acheivable and when (this latter is a hard problem). Cheers Chris Rose From jlai@ece.rutgers.edu Tue Feb 11 10:00:08 1997 Return-Path: Received: from zen.rutgers.edu by MOGLI.rutgers.edu (4.1/25-eef) id AA20028; Tue, 11 Feb 97 10:00:06 EST Received: from ece.rutgers.edu (ee.rutgers.edu [128.6.46.13]) by zen.rutgers.edu (8.6.12+bestmx+oldruq+newsunq/8.6.6) with SMTP id JAA12043 for ; Tue, 11 Feb 1997 09:43:17 -0500 Date: Tue, 11 Feb 97 9:43:39 EST From: Jie Lai To: Christopher Rose Subject: Re: test In-Reply-To: Your message of Mon, 10 Feb 97 21:59:52 EST Message-Id: Status: RO Hi, Dr. Rose, I go to the course web page. All the problem set is for 1996. Is that we supposed to do? Also, problem 2.13 is not on there. Is this you just add up? Just start to use this group. Thanks! Jie Lai From crose Tue Feb 11 11:49:49 1997 Return-Path: Received: by MOGLI.rutgers.edu (4.1/25-eef) id AA20122; Tue, 11 Feb 97 11:49:48 EST Date: Tue, 11 Feb 97 11:49:48 EST From: Christopher Rose Full-Name: Christopher Rose Message-Id: <9702111649.AA20122@MOGLI.rutgers.edu> To: jlai@ee.rutgers.edu Subject: Re: test Cc: crose Status: RO Yup, 2.13 is one I just want you to think about. And yes the problem sets are the same as last year. Cheers From jlai@liman.Rutgers.EDU Thu Feb 13 10:14:46 1997 Return-Path: Received: from ece.rutgers.edu by MOGLI.rutgers.edu (4.1/25-eef) id AA22494; Thu, 13 Feb 97 10:14:45 EST Received: from liman (liman.rutgers.edu [128.6.110.6]) by ece.rutgers.edu (8.6.12+bestmx+oldruq+newsunq/8.6.6) with ESMTP id JAA02294 for ; Thu, 13 Feb 1997 09:57:08 -0500 Received: from cell.winlab.rutgers.edu by liman (SMI-8.6/SMI-SVR4) id JAA03013; Thu, 13 Feb 1997 09:58:19 -0500 Received: from cell by cell.winlab.rutgers.edu (SMI-8.6/SMI-SVR4) id JAA05165; Thu, 13 Feb 1997 09:58:19 -0500 Sender: jlai@liman.Rutgers.EDU Message-Id: <33032C09.3828@winlab.rutgers.edu> Date: Thu, 13 Feb 1997 09:58:17 -0500 From: Jie Lai X-Mailer: Mozilla 3.0 (X11; I; SunOS 5.5.1 sun4m) Mime-Version: 1.0 To: ece542@ece.rutgers.edu Subject: Prob 2.12 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Status: RO For prob. 2.12, I got I(X;Y;Z) = sum{ p(XYZ)log[ (p(XY)p(XZ)p(YZ))/(p(X)p(Y)p(Z)p(XYZ)) ] } = sum{ p(XYZ)log[ (p(X|Y)p(X|Z))/(p(X)p(X|YZ)) ] } It seems a nice SYMMETRY form. From Venn diagram, it also looks ok. What I donot understand is when and how I(X;Y;Z) can get negative as the book said. Using the equation from the book: I(X;Y;Z) = I(X;Y) - I(X;Y|Z) = I(X;Y) - ( I(X;Y,Z) - I(X;Z) ) If X and Y are indep., thus I(X;Y)=0, Y has no mutual info. with X. It seems to me that I(X;Y,Z) will equal I(X;Z). So I(X;Y;Z)=0. How come it get negative ? Please help me get out! Anybody! Jie From crose@mogli.rutgers.edu Thu Feb 13 18:13:06 1997 Return-Path: Received: from mogli.rutgers.edu by MOGLI.rutgers.edu with UUCP (4.1/25-eef) id AA22857; Thu, 13 Feb 97 18:13:00 EST Full-Name: Christopher Rose Received: by mogli (4.1/SMI-4.1) id AA13552; Thu, 13 Feb 97 17:07:30 EST Date: Thu, 13 Feb 97 17:07:30 EST From: crose@mogli.rutgers.edu (Christopher Rose) Message-Id: <9702132207.AA13552@mogli> To: 330_542@MOGLI.rutgers.edu Cc: crose@MOGLI.rutgers.edu Status: RO QUESTION: From: Jie Lai X-Mailer: Mozilla 3.0 (X11; I; SunOS 5.5.1 sun4m) Mime-Version: 1.0 To: ece542@ece.rutgers.edu Subject: Prob 2.12 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit For prob. 2.12, I got I(X;Y;Z) = sum{ p(XYZ)log[ (p(XY)p(XZ)p(YZ))/(p(X)p(Y)p(Z)p(XYZ)) ] } = sum{ p(XYZ)log[ (p(X|Y)p(X|Z))/(p(X)p(X|YZ)) ] } It seems a nice SYMMETRY form. From Venn diagram, it also looks ok. What I donot understand is when and how I(X;Y;Z) can get negative as the book said. Using the equation from the book: I(X;Y;Z) = I(X;Y) - I(X;Y|Z) = I(X;Y) - ( I(X;Y,Z) - I(X;Z) ) If X and Y are indep., thus I(X;Y)=0, Y has no mutual info. with X. It seems to me that I(X;Y,Z) will equal I(X;Z). So I(X;Y;Z)=0. How come it get negative ? Please help me get out! Anybody! Jie From kangp@ms.com Thu Feb 13 23:59:04 1997 Return-Path: Received: from bkinis1-1.morgan.com (bkinis.ms.com) by MOGLI.rutgers.edu (4.1/25-eef) id AA23550; Thu, 13 Feb 97 23:59:03 EST Received: (from mail@localhost) by bkinis1-1.morgan.com (8.8.5/fw v1.22) id XAA29198 for ; Thu, 13 Feb 1997 23:43:01 -0500 (EST) Received: from bkadm1.morgan.com(138.20.76.11) by bkinis1-1.morgan.com via smap (V1.3) id sma029168; Thu Feb 13 23:42:48 1997 Received: from bkis167.morgan.com (bkis167.morgan.com [138.20.77.137]) by bkadm1.morgan.com (8.8.5/hub v1.44) with ESMTP id XAA15272 for ; Thu, 13 Feb 1997 23:42:48 -0500 (EST) From: Porlin Kang Received: (from kangp@localhost) by bkis167.morgan.com (8.8.5/client v1.15) id XAA13479 for crose@mogli.rutgers.edu; Thu, 13 Feb 1997 23:42:47 -0500 (EST) Date: Thu, 13 Feb 1997 23:42:47 -0500 (EST) Message-Id: <9702132342.ZM13477@ms.com> In-Reply-To: crose@mogli.rutgers.edu (Christopher Rose) "" (Feb 13, 5:07pm) References: <9702132207.AA13552@mogli> X-Mailer: Z-Mail (3.2.1 10oct95) To: crose@mogli.rutgers.edu (Christopher Rose) Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Status: RO I wish I could help .... I haven't got time to start looking into it yet !!! On Feb 13, 5:07pm, Christopher Rose wrote: > Subject: > > QUESTION: > > From: Jie Lai > X-Mailer: Mozilla 3.0 (X11; I; SunOS 5.5.1 sun4m) > Mime-Version: 1.0 > To: ece542@ece.rutgers.edu > Subject: Prob 2.12 > Content-Type: text/plain; charset=us-ascii > Content-Transfer-Encoding: 7bit > > For prob. 2.12, I got > > I(X;Y;Z) = sum{ p(XYZ)log[ (p(XY)p(XZ)p(YZ))/(p(X)p(Y)p(Z)p(XYZ)) ] } > > = sum{ p(XYZ)log[ (p(X|Y)p(X|Z))/(p(X)p(X|YZ)) ] } > > > It seems a nice SYMMETRY form. From Venn diagram, it also looks ok. > What I donot understand is when and how I(X;Y;Z) can get negative > as the book said. Using the equation from the book: > > I(X;Y;Z) = I(X;Y) - I(X;Y|Z) > = I(X;Y) - ( I(X;Y,Z) - I(X;Z) ) > > If X and Y are indep., thus I(X;Y)=0, Y has no mutual info. with X. > It seems to me that I(X;Y,Z) will equal I(X;Z). So I(X;Y;Z)=0. How > come it get negative ? > > Please help me get out! Anybody! > > Jie > >-- End of excerpt from Christopher Rose From crose Mon Feb 17 13:39:26 1997 Return-Path: Received: by MOGLI.rutgers.edu (4.1/25-eef) id AA27090; Mon, 17 Feb 97 13:39:20 EST Date: Mon, 17 Feb 97 13:39:20 EST From: Christopher Rose Full-Name: Christopher Rose Message-Id: <9702171839.AA27090@MOGLI.rutgers.edu> To: 330_542 Subject: problem 2.12 Status: RO Since I don't seem to have a record of anyone coming forward with the question about the negativity of I(X:Y:Z), here's a solution. Suppose X and Y are independent. Then as you point out, I(X:Y)=0. However, just because two variables are independen does not mean that they are CONDITIONALLY INDEPENDENT. You know, I think I'm repeating either the solution, or what one of you posted (Kabir)... let me check a moment... nope, it looks like I've killed the file! ARRRRGGGGG! In any case, consider z=x+y with x and y indep. Well given z, x and y are no longer independent so their mututal info is NOT ZERO. This means that I(X:Y:Z) is negative. Cheers, Chris Rose From jlai@ece.rutgers.edu Tue Feb 18 15:41:35 1997 Return-Path: Received: from zen.rutgers.edu by MOGLI.rutgers.edu (4.1/25-eef) id AA29011; Tue, 18 Feb 97 15:41:30 EST Received: from ece.rutgers.edu (ee.rutgers.edu [128.6.46.13]) by zen.rutgers.edu (8.6.12+bestmx+oldruq+newsunq/8.6.6) with SMTP id PAA24404 for <330_542@mogli.rutgers.edu>; Tue, 18 Feb 1997 15:23:19 -0500 Resent-Message-Id: <199702182023.PAA24404@zen.rutgers.edu> Received: from localhost (localhost) by zen.rutgers.edu (8.6.12+bestmx+oldruq+newsunq/8.6.6) with internal id MAA24155; Tue, 18 Feb 1997 12:59:23 -0500 Date: Tue, 18 Feb 1997 12:59:23 -0500 From: Mail Delivery Subsystem Subject: Returned mail: User unknown Message-Id: <199702181759.MAA24155@zen.rutgers.edu> To: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="MAA24155.856288763/zen.rutgers.edu" Resent-To: 330_542@mogli.rutgers.edu Resent-Date: Tue, 18 Feb 97 15:23:45 EST Resent-From: Jie Lai Status: RO This is a MIME-encapsulated message --MAA24155.856288763/zen.rutgers.edu The original message was received at Tue, 18 Feb 1997 12:59:22 -0500 from ee.rutgers.edu [128.6.46.13] ----- The following addresses had delivery problems ----- <330_542@ece> (unrecoverable error) ----- Transcript of session follows ----- ... while talking to ece.rutgers.edu.: >>> RCPT To:<330_542@ece> <<< 550 <330_542@ece>... User unknown 550 <330_542@ece>... User unknown ----- Original message follows ----- --MAA24155.856288763/zen.rutgers.edu Content-Type: message/rfc822 Return-Path: jlai Received: from ece.rutgers.edu (ee.rutgers.edu [128.6.46.13]) by zen.rutgers.edu (8.6.12+bestmx+oldruq+newsunq/8.6.6) with SMTP id MAA24154 for <330_542@ece>; Tue, 18 Feb 1997 12:59:22 -0500 Date: Tue, 18 Feb 97 12:59:47 EST From: Jie Lai To: 330_542@ece Subject: solution for hw Message-ID: Can we get the solution for HW, just for the first few sets, before the take-home exam ? So we could learn some techniques on how to think, formulate and solve the problems. Jie --MAA24155.856288763/zen.rutgers.edu-- From crose Tue Feb 18 17:19:28 1997 Return-Path: Received: by MOGLI.rutgers.edu (4.1/25-eef) id AA29103; Tue, 18 Feb 97 17:19:25 EST Date: Tue, 18 Feb 97 17:19:25 EST From: Christopher Rose Full-Name: Christopher Rose Message-Id: <9702182219.AA29103@MOGLI.rutgers.edu> To: 330_542 Subject: problem set solutions Cc: crose Status: RO Hi, the solutions to ps 1 and 2 are out on the web. The solutions to ps3 should arrive shortly (also on the web). Cheers, From crose Tue Feb 25 18:25:33 1997 Return-Path: Received: by MOGLI.rutgers.edu (4.1/25-eef) id AA07289; Tue, 25 Feb 97 18:25:32 EST Date: Tue, 25 Feb 97 18:25:32 EST From: Christopher Rose Full-Name: Christopher Rose Message-Id: <9702252325.AA07289@MOGLI.rutgers.edu> To: doolie@eden.rutgers.edu Subject: Hi Cc: crose@MOGLI.rutgers.edu Status: RO Hi Dong, Just ask the question on the mailing group first (330_452@mogli.rutgers.edu) and then set up a time with me if your questions are not answered. I'm at a meeting on thursday morning, but will be able to come in for an appointment if you'd like. Cheers Chris Rose From jlai@ece.rutgers.edu Wed Feb 26 17:41:29 1997 Return-Path: Received: from zen.rutgers.edu by MOGLI.rutgers.edu (4.1/25-eef) id AA09037; Wed, 26 Feb 97 17:41:29 EST Received: from ece.rutgers.edu (ee.rutgers.edu [128.6.46.13]) by zen.rutgers.edu (8.6.12+bestmx+oldruq+newsunq/8.6.6) with SMTP id RAA08031 for ; Wed, 26 Feb 1997 17:21:50 -0500 Date: Wed, 26 Feb 97 17:22:19 EST From: Jie Lai To: Christopher Rose Subject: Re: ready In-Reply-To: Your message of Mon, 24 Feb 97 10:48:20 EST Message-Id: Status: RO Hi, Dr. Rose, Q: For prob. 4, should we compile a program to count the "trans. prob." of all possible case? I think that means 27*27 possiblities. Do I understand the question correctly? Also there are 4 paragraphs, not 3. For paris of letters followed by pairs of letters, there are 27**4 possibilities, most of them are zeros, of course. Is that kind of prob. count we need to do and then calc. the entropy? Thanks. Jie From crose@mogli.rutgers.edu Wed Feb 26 21:45:29 1997 Return-Path: Received: from mogli.rutgers.edu by MOGLI.rutgers.edu with UUCP (4.1/25-eef) id AA09208; Wed, 26 Feb 97 21:44:01 EST Full-Name: Christopher Rose Received: by mogli (4.1/SMI-4.1) id AA02132; Wed, 26 Feb 97 21:13:17 EST Date: Wed, 26 Feb 97 21:13:17 EST From: crose@mogli.rutgers.edu (Christopher Rose) Message-Id: <9702270213.AA02132@mogli> To: jlai@ee.rutgers.edu Subject: Re: ready Cc: 330_542@MOGLI.rutgers.edu Status: RO That's the basic idea. Yes there are 27**4 possibilities, but it's pretty obvious from reading the three paragraphs taht there';s nowhere near those sorts of numbers. I'm asking for a "rough estimate" of the transition probs since there's not enough data to say anything. Nonetheless, if counting the doubles is too onerous, just count the relative frequencies of double letter combinations and enter them in a table. A computer program IS A GOOD IDEA. In fact, you can take some text from anywhere if you'd like (the larger the better) and do the relative frequency calculations. So consider this an ammendment to the problem. Cheers Chris Rose From kangp@ms.com Mon Mar 3 10:19:09 1997 Return-Path: Received: from bkinis1-1.morgan.com (bkinis.ms.com) by MOGLI.rutgers.edu (4.1/25-eef) id AA14498; Mon, 3 Mar 97 10:19:08 EST Received: (from mail@localhost) by bkinis1-1.morgan.com (8.8.5/fw v1.22) id KAA15080; Mon, 3 Mar 1997 10:00:02 -0500 (EST) Received: from bkadm1.morgan.com(138.20.76.11) by bkinis1-1.morgan.com via smap (V1.3) id sma014974; Mon Mar 3 09:59:45 1997 Received: from bkis167.morgan.com (bkis167.morgan.com [138.20.77.137]) by bkadm1.morgan.com (8.8.5/hub v1.44) with ESMTP id JAA21925; Mon, 3 Mar 1997 09:59:44 -0500 (EST) From: Porlin Kang Received: (from kangp@localhost) by bkis167.morgan.com (8.8.5/client v1.15) id JAA03286; Mon, 3 Mar 1997 09:59:44 -0500 (EST) Date: Mon, 3 Mar 1997 09:59:44 -0500 (EST) Message-Id: <9703030959.ZM3284@ms.com> In-Reply-To: crose@mogli.rutgers.edu (Christopher Rose) "Re: ready" (Feb 26, 9:13pm) References: <9702270213.AA02132@mogli> X-Mailer: Z-Mail (3.2.1 10oct95) To: crose@mogli.rutgers.edu (Christopher Rose) Subject: error ? Cc: 330_542@MOGLI.rutgers.edu Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Status: RO I think there is a typo in question 3. The transition prob out of state 0 doesn't sum up to 1 !! Should P = 0.5 instead of 0.25 ? 1|0 thanks From kangp@ms.com Mon Mar 3 10:30:56 1997 Return-Path: Received: from bkinis1-1.morgan.com (bkinis.ms.com) by MOGLI.rutgers.edu (4.1/25-eef) id AA14504; Mon, 3 Mar 97 10:30:53 EST Received: (from mail@localhost) by bkinis1-1.morgan.com (8.8.5/fw v1.22) id KAA18999; Mon, 3 Mar 1997 10:11:50 -0500 (EST) Received: from bkadm1.morgan.com(138.20.76.11) by bkinis1-1.morgan.com via smap (V1.3) id sma018788; Mon Mar 3 10:11:03 1997 Received: from bkis167.morgan.com (bkis167.morgan.com [138.20.77.137]) by bkadm1.morgan.com (8.8.5/hub v1.44) with ESMTP id KAA22802; Mon, 3 Mar 1997 10:11:03 -0500 (EST) From: Porlin Kang Received: (from kangp@localhost) by bkis167.morgan.com (8.8.5/client v1.15) id KAA03302; Mon, 3 Mar 1997 10:11:02 -0500 (EST) Date: Mon, 3 Mar 1997 10:11:02 -0500 (EST) Message-Id: <9703031011.ZM3300@ms.com> In-Reply-To: crose@mogli.rutgers.edu (Christopher Rose) "Re: ready" (Mar 3, 9:53am) References: <9703031453.AA05348@mogli> X-Mailer: Z-Mail (3.2.1 10oct95) To: crose@mogli.rutgers.edu (Christopher Rose) Subject: Re: ready Cc: 330_542@MOGLI.rutgers.edu Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Status: RO Prof Rose, Prob 2 is tough. Maybe i am too strongly influenced by the transition diagram. According to the book, not all agglomeration of state X yields a Markov Chain. So, an answer must exists :) But I can't see any relation between the transition prob and the past history. I am not familiar with all classes of Markov (aperiodic, irreducible, etc). Do I need more knowledge than discussed in the class to solve this problem ? Perhaps I should solve this problem from a totally different angle ? (i.e. not straight invocation of the Def of Markov) ?? thanks /pl From kangp@ms.com Mon Mar 3 10:33:14 1997 Return-Path: Received: from exinis1-1.morgan.com (exinis.ms.com) by MOGLI.rutgers.edu (4.1/25-eef) id AA14509; Mon, 3 Mar 97 10:33:12 EST Received: (from mail@localhost) by exinis1-1.morgan.com (8.8.5/fw v1.22) id KAA16248 for ; Mon, 3 Mar 1997 10:14:09 -0500 (EST) Received: from bkadm1.morgan.com(138.20.76.11) by exinis1-1.morgan.com via smap (V1.3) id sma016135; Mon Mar 3 10:13:47 1997 Received: from bkis167.morgan.com (bkis167.morgan.com [138.20.77.137]) by bkadm1.morgan.com (8.8.5/hub v1.44) with ESMTP id KAA22985 for ; Mon, 3 Mar 1997 10:13:46 -0500 (EST) From: Porlin Kang Received: (from kangp@localhost) by bkis167.morgan.com (8.8.5/client v1.15) id KAA03296 for crose@mogli.rutgers.edu; Mon, 3 Mar 1997 10:09:48 -0500 (EST) Date: Mon, 3 Mar 1997 10:09:48 -0500 (EST) Message-Id: <9703031009.ZM3294@ms.com> In-Reply-To: Porlin Kang "error ?" (Mar 3, 9:59am) References: <9702270213.AA02132@mogli> <9703030959.ZM3284@ms.com> X-Mailer: Z-Mail (3.2.1 10oct95) To: crose@mogli.rutgers.edu (Christopher Rose) Subject: Re: error ? Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Status: RO I solved this problem by assuming the typo and obtain an Entropy Rate as a function of k in part b. On Mar 3, 9:59am, Porlin Kang wrote: > Subject: error ? > I think there is a typo in question 3. > The transition prob out of state 0 doesn't sum up to 1 !! > Should P = 0.5 instead of 0.25 ? > 1|0 > > thanks >-- End of excerpt from Porlin Kang From crose Mon Mar 3 13:42:28 1997 Return-Path: Received: by MOGLI.rutgers.edu (4.1/25-eef) id AA14694; Mon, 3 Mar 97 13:42:27 EST Date: Mon, 3 Mar 97 13:42:27 EST From: Christopher Rose Full-Name: Christopher Rose Message-Id: <9703031842.AA14694@MOGLI.rutgers.edu> To: crose@mogli.rutgers.edu, kangp@ms.com Subject: Re: error ? Cc: crose@MOGLI.rutgers.edu Status: RO I'll check right now.... You're almost correct. p_0|0 should be 0.75 since there's no -1 state Thanks for catching that. Cheers, Chris Rose From crose Mon Mar 3 13:45:02 1997 Return-Path: Received: by MOGLI.rutgers.edu (4.1/25-eef) id AA14699; Mon, 3 Mar 97 13:44:56 EST Date: Mon, 3 Mar 97 13:44:56 EST From: Christopher Rose Full-Name: Christopher Rose Message-Id: <9703031844.AA14699@MOGLI.rutgers.edu> To: crose@mogli.rutgers.edu, kangp@ms.com Subject: Re: ready Cc: 330_542@MOGLI.rutgers.edu Status: RO Hi Porlin, I think all you need is the markov property. You have to find class(es) of original chains for which certain subdivisions will be markov as well. Yes, it can be a tough problem, but if you use the markov principle as the guide you should be able to identify at least one structure which always results in a new markov chain upon subdivision. Good luck! Chris Rose PS: And keep thepublic questions coming! From jlai@ece.rutgers.edu Fri Mar 14 14:36:38 1997 Return-Path: Received: from zen.rutgers.edu by MOGLI.rutgers.edu (4.1/25-eef) id AA01006; Fri, 14 Mar 97 14:36:38 EST Received: from ece.rutgers.edu (ee.rutgers.edu [128.6.46.13]) by zen.rutgers.edu (8.6.12+bestmx+oldruq+newsunq/8.6.6) with SMTP id OAA08675 for ; Fri, 14 Mar 1997 14:14:04 -0500 Date: Fri, 14 Mar 97 14:14:41 EST From: Jie Lai To: Christopher Rose Subject: Re: grading policy In-Reply-To: Your message of Fri, 14 Mar 97 13:59:05 EST Message-Id: Status: RO Hi, Dr. Rose, Is the channel capacity law we are talking about related to Shannon(?)'s original statement, like C = W log( 1 + SNR ) for Gaussian channel? How they are related? This will be covered in class, isn't it? Jie From crose Fri Mar 14 14:40:24 1997 Return-Path: Received: by MOGLI.rutgers.edu (4.1/25-eef) id AA01018; Fri, 14 Mar 97 14:40:23 EST Date: Fri, 14 Mar 97 14:40:23 EST From: Christopher Rose Full-Name: Christopher Rose Message-Id: <9703141940.AA01018@MOGLI.rutgers.edu> To: jlai@ee.rutgers.edu Subject: Re: grading policy Cc: crose Status: RO We've not gotten there yet, but it will be covered. We first have to learn a bit about continous entropy and taht happens later. Cheers, Chris Rose From kangp@ms.com Fri Mar 14 17:34:37 1997 Return-Path: Received: from exinis1-1.morgan.com (exinis.ms.com) by MOGLI.rutgers.edu (4.1/25-eef) id AA01290; Fri, 14 Mar 97 17:34:36 EST Received: (from mail@localhost) by exinis1-1.morgan.com (8.8.5/fw v1.22) id RAA09306 for ; Fri, 14 Mar 1997 17:13:33 -0500 (EST) Received: from hqmail1.morgan.com(144.14.61.155) by exinis1-1.morgan.com via smap (V1.3) id sma009284; Fri Mar 14 17:13:15 1997 Received: from bkis167.morgan.com (bkis167.morgan.com [138.20.77.137]) by hqmail1.morgan.com (8.8.5/hub v1.44) with ESMTP id RAA15941 for ; Fri, 14 Mar 1997 17:13:15 -0500 (EST) From: Porlin Kang Received: (from kangp@localhost) by bkis167.morgan.com (8.8.5/client v1.15) id RAA16691 for crose@MOGLI.rutgers.edu; Fri, 14 Mar 1997 17:13:14 -0500 (EST) Date: Fri, 14 Mar 1997 17:13:14 -0500 (EST) Message-Id: <9703141713.ZM16689@ms.com> In-Reply-To: Christopher Rose "grading policy" (Mar 14, 1:59pm) References: <9703141859.AA00896@MOGLI.rutgers.edu> X-Mailer: Z-Mail (3.2.1 10oct95) To: Christopher Rose Subject: Re: grading policy Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Status: RO Dr Chris, I was trying to ask you about this when we meet again. It seems that the number doesn't add up to the total. Is that intentional (I meant, does those numbers on each page meant anything ?) ? On the front page, you wrote 4 numbers that adds up to 130. However, on the following pages, the numbers were different. thanks /pl On Mar 14, 1:59pm, Christopher Rose wrote: > Subject: grading policy > Hi Folks, > > I encourage you to look over your exams and the solutions and question > why you were assigned the grades you were. When you get out in the > world and your boss says "good job" or "bad job" you should never just > take it at face value. ALWAYS question authority. > > In any case, here's the official policy on grading. If you had a > problem with a grade I assigned, you must write up your objection and > submit it to met along with the whole exam. I will look at the exam > and make a decision. Owing to time constraints, however, there will > only be one such appeals process. The grade is final after that. You > have until monday (the day we come back) to submit your requests for > regrade. > > So hop to it !!!! :) > > Cheers, > > Chris Rose >-- End of excerpt from Christopher Rose From crose Fri Mar 14 17:37:27 1997 Return-Path: Received: by MOGLI.rutgers.edu (4.1/25-eef) id AA01307; Fri, 14 Mar 97 17:37:26 EST Date: Fri, 14 Mar 97 17:37:26 EST From: Christopher Rose Full-Name: Christopher Rose Message-Id: <9703142237.AA01307@MOGLI.rutgers.edu> To: kangp@ms.com Subject: Re: grading policy Cc: crose@MOGLI.rutgers.edu Status: R Hi Porlin, Each question was worth 50 points. I threw numbers down more or less at random. However, there should be numbers which denote the oint totals for each problem somewhere (but maybe not). in any case, if you go through and add humbers I doubt you'll get the total on the front. I can take a look of course if you'd like. Just drop the exam off next time you come in (or just wait until monday when we get back). Cheers, Chris Rose From kangp@ms.com Wed Apr 9 13:46:46 1997 Return-Path: Received: from exinis1-1.morgan.com (exinis.ms.com) by MOGLI.rutgers.edu (4.1/25-eef) id AA00269; Wed, 9 Apr 97 13:46:42 EDT Received: (from mail@localhost) by exinis1-1.morgan.com (8.8.5/fw v1.22) id NAA28420 for ; Wed, 9 Apr 1997 13:21:20 -0400 (EDT) Received: from bkadm1.morgan.com(138.20.76.11) by exinis1-1.morgan.com via smap (V1.3) id sma028334; Wed Apr 9 13:20:49 1997 Received: from bkis167.morgan.com (bkis167.morgan.com [138.20.77.137]) by bkadm1.morgan.com (8.8.5/hub v1.44) with ESMTP id NAA10167 for ; Wed, 9 Apr 1997 13:20:47 -0400 (EDT) From: Porlin Kang Received: (from kangp@localhost) by bkis167.morgan.com (8.8.5/client v1.15) id NAA11855 for crose@MOGLI.rutgers.edu; Wed, 9 Apr 1997 13:20:46 -0400 (EDT) Date: Wed, 9 Apr 1997 13:20:46 -0400 (EDT) Message-Id: <9704091320.ZM11853@morgan.com> X-Mailer: Z-Mail (3.2.1 10oct95) To: crose@MOGLI.rutgers.edu Subject: 2 questions Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Status: RO Dr Rose, On page 99 of the text, there is a statement that says : (roughly in the middle) o The two longest codewords differ .... .... ... .... Now we can exchange the longest lenght codewords so the two lowest prob are associated with 2 siblings on the tree. This does not change the expected length .... My question is : how could the reassigment of probability with the codewords not change the expected length ? Shouldn't it be decreased ? ------------------ Somehow we skipped the proof of the Kraft's infinite source Uniquely decodable. Did you mentioned that you don't like the proof in the book ? thanks /pl From crose Wed Apr 9 17:59:53 1997 Return-Path: Received: by MOGLI.rutgers.edu (4.1/25-eef) id AA00648; Wed, 9 Apr 97 17:59:48 EDT Date: Wed, 9 Apr 97 17:59:48 EDT From: Christopher Rose Full-Name: Christopher Rose Message-Id: <9704092159.AA00648@MOGLI.rutgers.edu> To: 330_542 Subject: questions Cc: crose Status: R Dr Rose, On page 99 of the text, there is a statement that says : (roughly in the middle) o The two longest codewords differ .... .... ... .... Now we can exchange the longest lenght codewords so the two lowest prob are associated with 2 siblings on the tree. This does not change the expected length .... My question is : how could the reassigment of probability with the codewords not change the expected length ? Shouldn't it be decreased ? ------------------ Somehow we skipped the proof of the Kraft's infinite source Uniquely decodable. Did you mentioned that you don't like the proof in the book ? thanks ***************** Question 1: the two longest codewords are siblings which means they are the same length. Therefore swapping them changes nothing. Question 2: The proof is really simple, but it makes me uncomfortable because you're taking limits when you're not sure they exist. To clean is simple. We just show that if the limit approached a number larger than 1, then there must exist some N for which the finite sum is greater than 1 (an impossibility owing to Kraft) and we're done. From crose Fri Apr 11 13:11:30 1997 Return-Path: Received: by MOGLI.rutgers.edu (4.1/25-eef) id AA02962; Fri, 11 Apr 97 13:11:25 EDT Date: Fri, 11 Apr 97 13:11:25 EDT From: Christopher Rose Full-Name: Christopher Rose Message-Id: <9704111711.AA02962@MOGLI.rutgers.edu> To: 330_542 Subject: questions Cc: crose@MOGLI.rutgers.edu Status: R Prof. Rose, How could we explain 188, eqn 8.11 ? H(Y) = H(Y,E) where event E is {Y=e}. Seems like this equation is not used in the proof though. ******************* Remember that H(Y,E) = H(Y)+H(E|Y). Now what is H(E|Y) where E is the event that Y=e? And yes it is used in the proof since equation 8.12 is a direct application. Cheers From kangp@ms.com Fri Apr 11 13:24:30 1997 Return-Path: Received: from exinis1-1.morgan.com (exinis.ms.com) by MOGLI.rutgers.edu (4.1/25-eef) id AA03002; Fri, 11 Apr 97 13:24:28 EDT Received: (from mail@localhost) by exinis1-1.morgan.com (8.8.5/fw v1.22) id MAA11217 for ; Fri, 11 Apr 1997 12:58:47 -0400 (EDT) Received: from bkadm1.morgan.com(138.20.76.11) by exinis1-1.morgan.com via smap (V1.3) id sma011196; Fri Apr 11 12:58:22 1997 Received: from bkis167.morgan.com (bkis167.morgan.com [138.20.77.137]) by bkadm1.morgan.com (8.8.5/hub v1.44) with ESMTP id MAA22912 for ; Fri, 11 Apr 1997 12:58:14 -0400 (EDT) From: Porlin Kang Received: (from kangp@localhost) by bkis167.morgan.com (8.8.5/client v1.15) id MAA19578 for crose@MOGLI.rutgers.edu; Fri, 11 Apr 1997 12:58:07 -0400 (EDT) Date: Fri, 11 Apr 1997 12:58:07 -0400 (EDT) Message-Id: <9704111257.ZM19576@morgan.com> In-Reply-To: Christopher Rose "questions" (Apr 11, 1:11pm) References: <9704111711.AA02962@MOGLI.rutgers.edu> X-Mailer: Z-Mail (3.2.1 10oct95) To: Christopher Rose Subject: Re: questions Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Status: R Prof, I guess my intended question was : H(Y) = H(Y,E) ? thanks /pl On Apr 11, 1:11pm, Christopher Rose wrote: > Subject: questions > > Prof. Rose, > > How could we explain 188, eqn 8.11 ? > H(Y) = H(Y,E) where event E is {Y=e}. > > Seems like this equation is not used in the proof though. > > ******************* > > Remember that H(Y,E) = H(Y)+H(E|Y). > > Now what is H(E|Y) where E is the event > that Y=e? > > And yes it is used in the proof since equation 8.12 > is a direct application. > > Cheers >-- End of excerpt from Christopher Rose From crose Fri Apr 11 14:39:59 1997 Return-Path: Received: by MOGLI.rutgers.edu (4.1/25-eef) id AB03114; Fri, 11 Apr 97 14:39:55 EDT Date: Fri, 11 Apr 97 14:39:55 EDT From: Mail Delivery Subsystem Full-Name: Mail Delivery Subsystem Message-Id: <9704111839.AB03114@MOGLI.rutgers.edu> Subject: Returned mail: User unknown To: crose Status: R ----- Transcript of session follows ----- 550 335_542... User unknown 550 log... User unknown ----- Unsent message follows ----- Received: by MOGLI.rutgers.edu (4.1/25-eef) id AA03112; Fri, 11 Apr 97 14:39:55 EDT Date: Fri, 11 Apr 97 14:39:55 EDT From: Christopher Rose Full-Name: Christopher Rose Message-Id: <9704111839.AA03112@MOGLI.rutgers.edu> To: kangp@ms.com Subject: Re: questions Cc: 335_542 Please reread my statements AND question. I think you'll find you answer there. Cheers, *************** I guess my intended question was : H(Y) = H(Y,E) ? thanks /pl On Apr 11, 1:11pm, Christopher Rose wrote: > Subject: questions > > Prof. Rose, > > How could we explain 188, eqn 8.11 ? > H(Y) = H(Y,E) where event E is {Y=e}. > > Seems like this equation is not used in the proof though. > > ******************* > > Remember that H(Y,E) = H(Y)+H(E|Y). > > Now what is H(E|Y) where E is the event > that Y=e? > > And yes it is used in the proof since equation 8.12 > is a direct application. > > Cheers >-- End of excerpt from Christopher Rose From crose Fri Apr 11 14:44:08 1997 Return-Path: Received: by MOGLI.rutgers.edu (4.1/25-eef) id AA03142; Fri, 11 Apr 97 14:44:02 EDT Date: Fri, 11 Apr 97 14:44:02 EDT From: Christopher Rose Full-Name: Christopher Rose Message-Id: <9704111844.AA03142@MOGLI.rutgers.edu> To: 330_542 Subject: mistake in address Status: R Sent this to the wrong adress. sorry folks.. _______________ Please reread my statements AND question. I think you'll find you answer there. Cheers, *************** I guess my intended question was : H(Y) = H(Y,E) ? thanks /pl On Apr 11, 1:11pm, Christopher Rose wrote: > Subject: questions > > Prof. Rose, > > How could we explain 188, eqn 8.11 ? > H(Y) = H(Y,E) where event E is {Y=e}. > > Seems like this equation is not used in the proof though. > > ******************* > > Remember that H(Y,E) = H(Y)+H(E|Y). > > Now what is H(E|Y) where E is the event > that Y=e? > > And yes it is used in the proof since equation 8.12 > is a direct application. > > Cheers >-- End of excerpt from Christopher Rose From kangp@ms.com Fri Apr 11 14:56:40 1997 Return-Path: Received: from exinis1-1.morgan.com (exinis.ms.com) by MOGLI.rutgers.edu (4.1/25-eef) id AA03186; Fri, 11 Apr 97 14:51:37 EDT Received: (from mail@localhost) by exinis1-1.morgan.com (8.8.5/fw v1.22) id OAA27730 for ; Fri, 11 Apr 1997 14:20:56 -0400 (EDT) Received: from bkadm1.morgan.com(138.20.76.11) by exinis1-1.morgan.com via smap (V1.3) id sma027625; Fri Apr 11 14:20:16 1997 Received: from bkis167.morgan.com (bkis167.morgan.com [138.20.77.137]) by bkadm1.morgan.com (8.8.5/hub v1.44) with ESMTP id OAA05084 for ; Fri, 11 Apr 1997 14:20:15 -0400 (EDT) From: Porlin Kang Received: (from kangp@localhost) by bkis167.morgan.com (8.8.5/client v1.15) id OAA19880 for crose@MOGLI.rutgers.edu; Fri, 11 Apr 1997 14:20:14 -0400 (EDT) Date: Fri, 11 Apr 1997 14:20:14 -0400 (EDT) Message-Id: <9704111420.ZM19878@morgan.com> In-Reply-To: Christopher Rose "Re: questions" (Apr 11, 2:39pm) References: <9704111839.AA03112@MOGLI.rutgers.edu> X-Mailer: Z-Mail (3.2.1 10oct95) To: Christopher Rose Subject: Re: questions Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Status: R of course ... H(E|Y) = 0 given event E is {Y=e}. So, H(Y) = H(Y,E) thanks /pl On Apr 11, 2:39pm, Christopher Rose wrote: > Subject: Re: questions > Please reread my statements AND question. I think > you'll find you answer there. > Cheers, > > *************** > I guess my intended question was : > H(Y) = H(Y,E) ? > > thanks > /pl > > On Apr 11, 1:11pm, Christopher Rose wrote: > > Subject: questions > > > > Prof. Rose, > > > > How could we explain 188, eqn 8.11 ? > > H(Y) = H(Y,E) where event E is {Y=e}. > > > > Seems like this equation is not used in the proof though. > > > > ******************* > > > > Remember that H(Y,E) = H(Y)+H(E|Y). > > > > Now what is H(E|Y) where E is the event > > that Y=e? > > > > And yes it is used in the proof since equation 8.12 > > is a direct application. > > > > Cheers > >-- End of excerpt from Christopher Rose > > >-- End of excerpt from Christopher Rose From crose Fri Apr 11 14:30:37 1997 Return-Path: Received: by qbeast.rutgers.edu (4.1/SMI-4.1) id AA10416; Fri, 11 Apr 97 14:30:36 EDT Date: Fri, 11 Apr 97 14:30:36 EDT From: crose (Christopher Rose) Message-Id: <9704111830.AA10416@qbeast.rutgers.edu> To: kangp@ms.com Subject: Re: questions Cc: crose Status: R Yup! From seupark@paul.rutgers.edu Mon Apr 14 19:36:16 1997 Return-Path: Received: from paul.rutgers.edu by MOGLI.rutgers.edu (4.1/25-eef) id AA06927; Mon, 14 Apr 97 19:36:10 EDT Received: from paul.rutgers.edu (seupark@localhost [127.0.0.1]) by paul.rutgers.edu (8.8.4/8.8.4) with SMTP id TAA09574; Mon, 14 Apr 1997 19:09:55 -0400 (EDT) Sender: seupark@paul.rutgers.edu Message-Id: <3352B942.3F51@paul.rutgers.edu> Date: Mon, 14 Apr 1997 19:09:54 -0400 From: Seungyoung Park Organization: Rutgers University X-Mailer: Mozilla 3.0 (X11; I; SunOS 5.5.1 sun4m) Mime-Version: 1.0 To: crose@MOGLI.rutgers.edu Cc: 330_542@MOGLI.rutgers.edu Subject: questions ? Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Status: R Dear Professor, I have some questions. 1) On page 193, eqn 8.29, --where I(.) is the indicator function. ; What is the indicatior function ? Could you give me some examples of indicator function ? 2) On page 198, Theorem 8.7.1(The channel coding theorem) -- there exists a sequence of (2nR, n) codes with - - ; a sequence of (2nR, n) codes, it means a code ? Is it right ? In the proof, it is eqn 8.62. Do I understand rightly ? 3) On page 227, Theoren 9.2.3 -- , to first order in the exponent. ; What is the "first order in the exponent" ? Could you explain it with some examples ? 4) On page 239, eqn 10.2, power constraint. ; I don't understand xi(codeword)2. If the code is the code of example 5.1.1(page 79), What is xi2 in this case ? In example, C(2) = 10, (10)2 = ? = 100 ? Thanks. From crose Mon Apr 14 19:47:13 1997 Return-Path: Received: by MOGLI.rutgers.edu (4.1/25-eef) id AA06945; Mon, 14 Apr 97 19:47:10 EDT Date: Mon, 14 Apr 97 19:47:10 EDT From: Christopher Rose Full-Name: Christopher Rose Message-Id: <9704142347.AA06945@MOGLI.rutgers.edu> To: seupark@paul.rutgers.edu Subject: Re: questions ? Cc: 330_542 Status: R 1) and indicator function is 1 when the "indicated condition" is true and zero otherwise. 2) A sequence of codes is what is meant. You're looking for codes which may vary with n. That is, there's a different code for each n. HOwever, the coding procedure (typical coding) does not change. 3) FIrst order in exponent means to first order in n. 4) x_i(w)^2 is the i^th power level used in sequence of n channel uses. Remember that for continuous channels we're using continous levels as opposed to binary symbols. Thus, the example on page 79, which is for discrete sources is not applicable. Think of it this way. For the continuous channel we're sending a signal during a channel use (say on an interval (0,T)). We can sent a large signal (with large power) and correspondingly large x_i or we can send a smaller signal. That is, we're sending actual levels over the channel. The only place where discreteness comes in is that we're using sequences of these powers to code discrete messages. Specifically, we have 2^(nR) messages which we're trying to code with a sequence of n levels (x_i). Cheers From oekelly@winlab.rutgers.edu Mon Apr 14 23:24:29 1997 Return-Path: Received: from liman (liman.rutgers.edu) by MOGLI.rutgers.edu (4.1/25-eef) id AA07161; Mon, 14 Apr 97 23:24:29 EDT Received: from liman1.winlab.rutgers.edu by liman (SMI-8.6/SMI-SVR4) id WAA23899; Mon, 14 Apr 1997 22:58:14 -0400 Received: by liman1.winlab.rutgers.edu (SMI-8.6/SMI-SVR4) id WAA13690; Mon, 14 Apr 1997 22:58:13 -0400 Date: Mon, 14 Apr 1997 22:58:13 -0400 From: oekelly@winlab.rutgers.edu (Owen Ernest Kelly) Message-Id: <199704150258.WAA13690@liman1.winlab.rutgers.edu> To: crose@MOGLI Subject: Re: talk Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Md5: 1FFuDklEwi9j/KogFcg23Q== Status: R The best I can say "huge". She seems to really really have the big picture. Her basic point was that source coding + channel coding + modulation should all be designed *jointly*. I could have told you THAT, but it seems overwhelmingly difficult. She has just given it the college try anyway and come away a winner. Her talk examined when you can get away with isolated designs and when you can't. Very impressive. If I had an NSF career award to give away I wouldn't hesitate to give it to her. Another amazing result was that if you have both power control and rate control, then the proper way to combat fading is to adjust your *rate* and leave power constant. To be more formal, if you compare capacity curves for i) shannon capacity ii) adjustable rate, adjustable power iii) adjustable rate, fixed power iv) fixed rate, adjustable power then iii) is much closer to i) than iv). That is, you take really big capacity hit if you fix the rate. -Owen From jlai@ece.rutgers.edu Mon Apr 28 11:58:50 1997 Return-Path: Received: from ece.rutgers.edu by MOGLI.rutgers.edu (4.1/25-eef) id AA19299; Mon, 28 Apr 97 11:58:42 EDT Received: (from jlai@localhost) by ece.rutgers.edu (8.6.12+bestmx+oldruq+newsunq/8.6.6) id LAA07296 for 330_542@mogli; Mon, 28 Apr 1997 11:28:00 -0400 Date: Mon, 28 Apr 97 11:28:00 EDT From: Jie Lai To: 330_542@mogli.rutgers.edu Subject: dual code Cd ? Message-Id: Status: R Dr. Rose, On page 76 of handout, there are sth. about linear code C and its dual code Cd. My question is, what is the defination of dual code Cd, how is it got from C, is there any relations between their generator and parity check matrix ? Thanks. Jie