Recap for Exam 3

(GUIs)
Searching (linear vs binary)
Sorting
- In particular: bubble, insertion, selection, merge, quick, heap
- recognize them in action (how do they work)
- compare and contrast (big-O, pre-sorted or not, how they work, etc.)
- Remember: O(1) < O(lg n) < O(n) < O(n lg n) < O(n^2) < O(n^3)
Trees
- Kinds and terminology: binary, BST, heaps; full, perfect, complete; child, leaf, root; etc.
- Traversals: in/pre/post order
- Searching: DFS vs BFS (stack vs queue)
- BST: adding, retrieving, and removing; using a BST ADT; performance
- Heaps / priority queues: adding, removing; max vs min order; using a PrQ ADT; performance
- (Huffman? Maybe in EC)

Integers as binary

dec     binary    hex     oct
65 == 01000001 == 0x41 == 0101

Different human notations for the same value
Java 7 binary literal: 0b0100_0001 (underscore optional)
In memory, it's all binary
however, interpretation of that binary sequence depends on data type

Integers: 2s complement, sign bit

0     int: 00000000 00000000 00000000 00000000   byte: 00000000
-1    int: 11111111 11111111 11111111 11111111   byte: 11111111
127   int: 00000000 00000000 00000000 01111111   byte: 01111111
-128  int: 11111111 11111111 11111111 10000000   byte: 10000000
128   int: 00000000 00000000 00000000 10000000   byte: --

Encoding: Characters as Binary

ASCII (or Unicode UTF-8)

ltr   binary    hex  dec
'A'  01000001  0x41  65
'a'  01100001  0x61  97
'b'  01100010  0x62  98
'0'  00110000  0x30  48
'1'  00110001  0x31  49
'2'  00110010  0x32  50
'3'  00110011  0x33  51

8 bits per byte; 1 byte = 1 symbol; 256 possible symbols.

int ltr = 'A';
System.out.println(ltr);  //prints 65

Customized Encoding

Suppose we had special need: Only need 211 assignment name abbreviations.
A, a, b, 0, 1, 2, ... 8, 9.
13 symbols) -> more than 8, less than 16 = 2^4 bits
```
'0'  0000
'1'  0001
'2'  0010
'3'  0011
...
'9'  1001
'A'  1100
'a'  1101
'b'  1110
```
Now only need 4 bits per symbol. More efficient encoding!

Entropy Encoding

From information theory: based on expected value of message
If certain symbols more likely (expected), we can build on that and use fewer bits for those
Variable length encoding
From Assignment names, suppose these ave frequencies: A = 1/2, 0 = 1/4, 1 = 1/8, 2 thru 9 = 1/64 each (for simple example)
```
A  0
0  10
1  110
2  111000
3  111001
4  111010
...
9  111111
```
Average bits per symbol: 1/2*1 + 1/4*2 + 1/8*3 + 8/64*8 = 19/8 = 2.375 bits/symbol
If all equally likely in message to encode: (8*6 + 3 + 2 + 1)/11 = 54/11 = ~5 bits/symbol
Prefix-free code: no complete symbol overlaps (is a prefix) for another

Huffman Coding

Method of building + using a binary tree to find variable-length prefix-free codes for compression
Compression: fewer bits needed to store/send the same data (provided some symbols more are freq.)
Do this because we implicitly build on frequency of the symbols in the data
Overhead: also need to send tree details (ie, symbol -> binary sequence translation details)
- Can use standard so don't have to transmit: such as based on frequency of letter use in English text

Building the Tree

Loop through your data and count the frequency of each symbol (similar to A02)
Then, create a leaf node for each symbol, including its count
Load nodes into min priority queue ordered by count
- Add nodes with a dummy value and count of 0 until at least 2 nodes in PQ
While PQ has more than one node:
- grab next two nodes (smallest counts) from PQ
- make them leaves of a new inner node, which has count == to sum of two subtree counts
- push inner node back into PQ
Remaining node is root of Huffman tree. (Note: always a full tree)

Example (symbol:count): A:32, 0:16, 1:8, 2:1, 3:1, 4:1, 5:1, 6:1; 7:1, 8:1, 9:1.
(Note: tree can vary depending on your policy of child order as they come out of PQ)
You try. Build tree from: a:7, b:5, c:4, d:2, e:1.
What are the paths (0 = left, 1 = right) to each node?

Using the Tree: Encoding

So, once you have the tree, run a DFS/traversal
Whenever you hit a leaf, save the symbol/path pair
- We'll talk about Maps next week: perfect for this
- Even without, you could just have a list of custom objects: symbol & path fields
Then, when writing message, loop up the symbol and corresponding path bits instead of symbol
Try it with your tree: cade

Using the Tree: Decoding

Given a bit stream, just traverse the tree (0 = left, 1 = right)
- When you hit a leaf, print/save the symbol, and go back to root
Given your tree, decode this into symbols: 01100100011
And this: 000101

Sending the Tree

To decode, you need the tree, but tree is specific to each data set
So we have to send the tree along with the data... preferably in a compact way.
- (again: advantage of a standard/preset tree on both ends)
The technique we'll use for next assignment (of course there are others)
- Do an pre-order (left-to-right) traversal.
- For each internal node, write a 0
- For each leaf node, write a 1 followed by symbol in 8 bits
  'a' = 01100001, 'b' = 01100010, 'c' = 01100011, etc.
Write out your tree. (Instead of 01100001, you can just write [a] for now.)

Receiving the Tree

Okay, now you'll receive the tree as a string of binary digits... how to rebuild it?
One way:

  create a stack
  read first 0 from stream
  set root to a new empty internal node and push it onto stack
  while stack is not empty:
      read next bit from stream
      if bit is 0:
          create a new empty internal node
      else on a 1:
          read 8 more bits and create a new leaf node with the data
      if stack.peek() does not have left child:
          add the new node as left child of stack.peek()
      else:
          add the new node as right child of stack.peek()
          pop the top of the stack (because now it has two children)
      if the new node is an internal node:
          push the new node onto stack

You try: 01[c]01[d]1[b]

Deep breath...

That's the theory.
So, to encode:
- Run through your data and count what you have
- Use the counts and a priority queue to build a Huffman tree
- Traverse the tree to determine paths (bit sequences) for each symbol (leaf node)
- Encode a copy of the tree into a file
- Append an encoding of all your data
On the other end, to decode:
- Decode tree using a stack
- Read in remaining data, using tree to translate back into symbols.

Technical details remaining

So encoded Huffman file is a long string of bits...
Files need to be a number of bytes, so how many bits to read of last byte?
- We'll write the symbol count as the first thing in the file to handle this
How to read in bytes and bits from a file?

Reading bytes from a file

Recommended: java.io.FileInputStream

String filename = ...
try {
  FileInputStream file = new FileInputStream(filename);
  int val = file.read();
  if (val == -1) {
    //reached end of file 
  }else {
    byte b = (byte) val;  //one way to convert
  }
}catch (IOException e) {
  System.out.println("Could not read from file: " + e.getMessage());
}

What about getting bits out of a byte?

[Out of time; will return to this on Wed.]

Bit operators: >>, <<, >>>, ~, &, |, ^
Numbers default to int data type, so work there when you can
ints are always signed in Java (no unsigned ints; bit twiddling is easier in C)
Masking

Examples:

  byte[] data = {1, 2, 3, 4};
  int together = 0;
  for (byte b : data) {
    together = (together << 8) | b;
  }
  System.out.println(together); //0x01020304 = 16909060

  byte b = '3';  // or 51 or 0x33
  for (int i = 0; i < 8; i++) {  //prints: 00110011
    int mask = 0x0080 >> i;
    int bit = b & mask;
    bit = bit >> (7 - i);
    System.out.print(bit);
  }
  System.out.println();

This will be covered in more detail in lab.

For next time...

A10 to be posted over weekend
Make sure you review the quizzes
- (and the slides and the assignments too, of course)
Exam 3 on Monday
Last chance for second EC codingbat tally is next Wednesday.

13b: Huffman Trees

ICS211, Spring 2013
Dr. Zach

Status check

Recap for Exam 3

Integers as binary

Encoding: Characters as Binary

Customized Encoding

Entropy Encoding

Huffman Coding

Building the Tree

Using the Tree: Encoding

Using the Tree: Decoding

Sending the Tree

Receiving the Tree

Deep breath...

Technical details remaining

Reading bytes from a file

What about getting bits out of a byte?

For next time...

13b: Huffman Trees

ICS211, Spring 2013Dr. Zach

Status check

Recap for Exam 3

Integers as binary

Encoding: Characters as Binary

Customized Encoding

Entropy Encoding

Huffman Coding

Building the Tree

Using the Tree: Encoding

Using the Tree: Decoding

Sending the Tree

Receiving the Tree

Deep breath...

Technical details remaining

Reading bytes from a file

What about getting bits out of a byte?

For next time...

ICS211, Spring 2013
Dr. Zach