Novel coding inequalities for mean codeword length and generalized entropy using noiseless communication

Aakanksha  Dwivedi; RAM NARESH SARASWAT

doi:10.5269/bspm.78359

Aakanksha Dwivedi Manipal University Jaipur
RAM NARESH SARASWAT Manipal University Jaipur

Resumo

Shannon’s entropy forms the basis for almost every aspect of information theory. It formulates the foundational stone for various source coding theorems assuming statistical independence and extensive systems. This research aims to investigate the possibility of deriving novel entropy measures using noiseless coding theorem. The obtained results find a widespread application in information theory and applied mathematics. To accomplish this, a novel expression for mean codeword length has been illustrated. Besides, established relation between entropy measure and its corresponding codeword length. The results obtained pave the way for a new avenue for entropy-based coding in non-extensive and information-rich environments.

Downloads

Não há dados estatísticos.

Referências

Shannon, C. E., A mathematical theory of communication, Bell Syst. Tech. J., 27, 379–423, 623–656, (1948).

Garcıa-Gonzalez, W., Flores-Fuentes, W., Sergiyenko, O., Rodrıguez-Quinonez, J. C., Miranda-Vega, J. E., Hernandez-Balbuena, D., Shannon entropy is used for feature extraction of optical patterns in the context of structural health monitoring, Entropy, 25(8), 1207, (2023).

Kraft, L. G., A device for quantizing, grouping and coding amplitude-modulated pulses, Doctoral dissertation, Massachusetts Institute of Technology, Cambridge, (1949).

Feinstein, A., Foundation of information theory, McGraw-Hill, New York, (1956).

Campbell, L. L., A coding theorem and Renyi’s entropy, Inf. Control, 8, 423–429, (1965).

Renyi, A., On measures of entropy and information, Proc. 4th Berkeley Symp. Math. Statist. Prob., 1, 547–561, (1961).

Kieffer, J. C., Variable-length source coding with a cost depending only on the codeword length, Inf. Control, 41, 136–146, (1979).

Jelinek, F., Buffer overflow in variable-length coding of fixed-rate sources, IEEE Trans. Inf. Theory, 3, 490–501, (1980).

Hooda, D. S., Bhaker, U. S., A generalized ‘useful’ information measure and coding theorems, Soochow J. Math., 23, 53–62, (1997).

Tsallis, C., The nonadditive entropy Sq and its applications in physics and elsewhere: some remarks, Entropy, 13, 1765–1804, (2011).

Nath, P., On a coding theorem connected with Renyi’s entropy, Inf. Control, 29(3), 234–242, (1975).

Belis, M., Guiasu, S., A quantitative-qualitative measure of information in cybernetic systems, IEEE Trans. Inf. Theory, 14(4), 593–594, (1968).

Longo, G., A noiseless coding theorem for sources having utilities, SIAM J. Appl. Math., 30(4), 739–748, (1976).

Guiasu, S., Picard, C. F., Borne inferieure de la longueur de certains codes, C. R. Acad. Sci. Paris, 273(A), 248–251, (1971).

Gurdial, P. F., On ‘useful’ information of order α, J. Combin. Inf. Syst. Sci., 2, 158–162, (1977).

Bhatia, P. K., Useful inaccuracy of order α and 1.1 coding, Soochow J. Math., 21(1), 81–87, (1995).

Tenaja, H., Hooda, D. S., Tuteja, R. K., Coding theorems on a generalized ‘useful’ information, Soochow J. Math., 11, 123–131, (1985).

Khan, A. B., Bhat, B. A., Pirzada, S., Some results on a generalized ‘useful’ information measure, J. Inequal. Pure Appl. Math., 6(4), 117, (2005).

Bhat, A. H., Baig, M. A. K., Characterization of new two-parametric generalized useful information measure, J. Inf. Sci. Theory Pract., 4(4), 64–74, (2016).

Bhat, A. H., Baig, M. A. K., New generalized measure of entropy of order α and type β and its coding theorems, Int. J. Inf. Sci. Syst., 5, 1–7, (2016).

Bhat, A. H., Baig, M. A. K., Noiseless coding theorem on the new generalized useful information measure of order α and type β, Asian J. Fuzzy Appl. Math., 4(6), 73–85, (2016).

Bhat, A. H., Baig, M. A. K., Some coding theorems on generalized Renyi’s entropy of order α and type β, Int. J. Appl. Math. Inf. Sci. Lett., 5(1), 13–19, (2017).

Jain, P., Tuteja, R. K., On coding theorem connected with ‘useful’ entropy of order β, Int. J. Math. Math. Sci., 12, 193–198, (1989).

Havrda, J., Charvat, F., Quantification method of classification processes: the concept of structural entropy, Kybernetika, 3, 30–35, (1967).

Mitter, J., Mathur, Y. D., Comparison of entropies of power distribution, ZAMM, 52, 239–240, (1972).

Srivastava, H. M., Agarwal, R., Jain, S., Integral transform and fractional derivative formulas involving the extended generalized hypergeometric functions and probability distributions, Math. Methods Appl. Sci., 40(1), 255–273, (2017).

Tsallis, C., Possible generalization of Boltzmann–Gibbs statistics, J. Stat. Phys., 52, 479, (1988).