Saltar para o conteúdo
Nydus Open Learning

Mathematics

Mathematics · Avançado · Nível 4

Information theory

Entropy, relative entropy and mutual information with their inequalities; source coding from Kraft's inequality through Huffman, typical sets and arithmetic coding; discrete and Gaussian channels, capacity and the channel coding theorem with its converse; linear block codes, syndrome decoding, Hamming and Reed-Solomon codes and the bounds that limit them.

Ages 18+ 44 lessons about 15.5 hours in all

Cria uma conta para começar Rever as bases

Podes começar já, seja o que for que tenhas feito antes. Não é preciso terminar primeiro nada desta página. Podes fazer uma pausa quando quiseres; o teu progresso é guardado automaticamente.

As lições

Podes abrir qualquer lição agora, por qualquer ordem. O plano recomenda uma; não bloqueia as restantes.

  1. 1 Information and entropy
  2. 2 The binary entropy function
  3. 3 Bounds on entropy
  4. 4 Jensen's inequality
  5. 5 Joint and conditional entropy
  6. 6 The chain rule for entropy
  7. 7 Relative entropy
  8. 8 Gibbs' inequality and cross-entropy
  9. 9 Mutual information
  10. 10 Identities and bounds for mutual information
  11. 11 The data processing inequality
  12. 12 Fano's inequality
  13. 13 Codes and prefix codes
  14. 14 Kraft's inequality
  15. 15 The source coding theorem
  16. 16 Shannon codes
  17. 17 Huffman's algorithm
  18. 18 Optimality of Huffman codes
  19. 19 The asymptotic equipartition property
  20. 20 Typical sets
  21. 21 Entropy rate
  22. 22 Arithmetic coding
  23. 23 Discrete memoryless channels
  24. 24 Channel capacity
  25. 25 The binary symmetric channel
  26. 26 The binary erasure channel
  27. 27 Symmetric channels
  28. 28 Computing capacities
  29. 29 The channel coding theorem
  30. 30 Random coding and joint typicality
  31. 31 The converse and feedback
  32. 32 Source-channel separation
  33. 33 The Gaussian channel
  34. 34 Bandwidth and the Shannon-Hartley law
  35. 35 Block codes
  36. 36 Hamming distance and minimum distance
  37. 37 Linear codes and generator matrices
  38. 38 Parity-check matrices
  39. 39 Syndrome decoding
  40. 40 Hamming codes
  41. 41 The Hamming bound
  42. 42 The Singleton bound and MDS codes
  43. 43 Cyclic codes
  44. 44 Reed-Solomon codes