Android Question .Net GZip Compression and B4A Decompression Problem

Ohanian

Active Member
Licensed User
Longtime User
Hi,

i'm trying to compress/decompress data from an asp.net web service to a b4a client.

here's the .net part :
B4X:
Context.Response.Expires = -1
'Context.Response.ContentType = "application/x-gzip"
Context.Response.ContentType = "application/json"
Context.Response.ContentEncoding = System.Text.Encoding.UTF8
Context.Response.Write(GZIPCompress("Hello, This is a sample text."))

 Function GZipCompress(ByVal s As String) As String
        Dim bytes = Encoding.Unicode.GetBytes(s)
        Using msi = New MemoryStream(bytes)
            Using mso = New MemoryStream()
                Using gs = New GZipStream(mso, CompressionMode.Compress)
                    msi.CopyTo(gs)
                End Using
                'Return Convert.ToBase64String(mso.ToArray())
                Return Encoding.UTF8.GetString(mso.ToArray)
            End Using
        End Using
    End Function

and b4a part :
B4X:
Dim sData As String   
        Dim cs As CompressedStreams
        Dim In As InputStream
        In = cs.WrapInputStream(Job.GetInputStream, "gzip")       
        Dim Reader As TextReader
        Reader.Initialize(In)
        sData = Reader.ReadAll
       
        Log(sData)

but got errors like these :
java.io.IOException: unknown format (magic number 7364)
java.io.IOException: unknown format (magic number 3448)
java.io.IOException: unknown format (magic number ef1f)

any suggestion?
 

Erel

B4X founder
Staff member
Licensed User
Longtime User
any suggestion?
Use B4J to implement the server side. It will be much simpler.

This line is wrong:
B4X:
Return Encoding.UTF8.GetString(mso.ToArray)
You cannot treat raw bytes as a UTF8 string. You need to either send the raw bytes or convert them to a string with base 64 encoding.
 
Upvote 0

Ohanian

Active Member
Licensed User
Longtime User
Hi,

Use B4J to implement the server side. It will be much simpler.

it's a part of a bigger project, so it's not possible to change the server side.

I've got 3 different results from .net side!

B4X:
                Dim sData As String = String.Empty
                sData = GZIPCompress("Hello, This is a sample text.")
                sData = CompressString("Hello, This is a sample text.")
                sData = Zip("Hello, This is a sample text.")

 Function GZipCompress(ByVal s As String) As String
        Dim bytes = Encoding.Unicode.GetBytes(s)
        Using msi = New MemoryStream(bytes)
            Using mso = New MemoryStream()
                Using gs = New GZipStream(mso, CompressionMode.Compress)
                    msi.CopyTo(gs)
                End Using
                Return Convert.ToBase64String(mso.ToArray())
                'Return Encoding.UTF8.GetString(mso.ToArray)
            End Using
        End Using
    End Function

 Function CompressString(ByVal text As String) As String
        Dim buffer__1 As Byte() = Encoding.UTF8.GetBytes(text)
        Dim memoryStream = New MemoryStream()
        Using gZipStream = New GZipStream(memoryStream, CompressionMode.Compress, True)
            gZipStream.Write(buffer__1, 0, buffer__1.Length)
        End Using

        memoryStream.Position = 0

        Dim compressedData = New Byte(memoryStream.Length - 1) {}
        memoryStream.Read(compressedData, 0, compressedData.Length)

        Dim gZipBuffer = New Byte(compressedData.Length + 3) {}
        Buffer.BlockCopy(compressedData, 0, gZipBuffer, 4, compressedData.Length)
        Buffer.BlockCopy(BitConverter.GetBytes(buffer__1.Length), 0, gZipBuffer, 0, 4)
        Return Convert.ToBase64String(gZipBuffer)
    End Function

 Public Shared Function Zip(ByVal text As String) As String
        Dim buffer As Byte() = System.Text.Encoding.Unicode.GetBytes(text)
        Dim ms As New MemoryStream()
        Using zipStream As New System.IO.Compression.GZipStream(ms, System.IO.Compression.CompressionMode.Compress, True)
            zipStream.Write(buffer, 0, buffer.Length)
        End Using

        ms.Position = 0
        Dim outStream As New MemoryStream()

        Dim compressed As Byte() = New Byte(ms.Length - 1) {}
        ms.Read(compressed, 0, compressed.Length)

        Dim gzBuffer As Byte() = New Byte(compressed.Length + 3) {}
        System.Buffer.BlockCopy(compressed, 0, gzBuffer, 4, compressed.Length)
        System.Buffer.BlockCopy(BitConverter.GetBytes(buffer.Length), 0, gzBuffer, 0, 4)
        Return Convert.ToBase64String(gzBuffer)
    End Function

but none of them are decompressed in b4a!

B4X:
Dim sData As String   
    Dim cs As CompressedStreams
    Dim decompressed() As Byte
   
    decompressed = cs.DecompressBytes("H4sIAAAAAAAEAPNgSGXIAcJ8Bh0GBYYQhgyGTIZiIAtCJgJxMZDMZSgAqkkF8kqAZAWQ1GMAAANbXTE6AAAA".GetBytes("UTF8"), "gzip")
    decompressed = cs.DecompressBytes("HQAAAB+LCAAAAAAABADzSM3JyddRCMnILFYAokSF4sTcgpxUhZLUihI9AHKPbswdAAAA".GetBytes("UTF8"), "gzip")
    decompressed = cs.DecompressBytes("OgAAAB+LCAAAAAAABADzYEhlyAHCfAYdBgWGEIYMhkyGYiALQiYCcTGQzGUoAKpJBfJKgGQFkNRjAAADW10xOgAAAA==".GetBytes("UTF8"), "gzip")
   
    sData = BytesToString(decompressed,0, decompressed.Length, "UTF8")
    Log(sData)

but got errors like this :
java.io.IOException: unknown format (magic number 674f)
 
Upvote 0

Ohanian

Active Member
Licensed User
Longtime User
You need to use StringUtils to convert the base64 string to bytes.

the same errors :

B4X:
Dim su As StringUtils
    Try       
        decompressed = cs.DecompressBytes(su.DecodeBase64("H4sIAAAAAAAEAPNgSGXIAcJ8Bh0GBYYQhgyGTIZiIAtCJgJxMZDMZSgAqkkF8kqAZAWQ1GMAAANbXTE6AAAA"), "gzip")
        sData = BytesToString(decompressed,0, decompressed.Length, "UTF8")
        Log(sData)       
    Catch
        Log(LastException)
    End Try
    Try
        decompressed = cs.DecompressBytes(su.DecodeBase64("HQAAAB+LCAAAAAAABADzSM3JyddRCMnILFYAokSF4sTcgpxUhZLUihI9AHKPbswdAAAA"), "gzip")
        sData = BytesToString(decompressed,0, decompressed.Length, "UTF8")
        Log(sData)       
    Catch
        Log(LastException)
    End Try
    Try
        decompressed = cs.DecompressBytes(su.DecodeBase64("OgAAAB+LCAAAAAAABADzYEhlyAHCfAYdBgWGEIYMhkyGYiALQiYCcTGQzGUoAKpJBfJKgGQFkNRjAAADW10xOgAAAA=="), "gzip")
        sData = BytesToString(decompressed,0, decompressed.Length, "UTF8")
        Log(sData)       
    Catch
        Log(LastException)
    End Try
B4X:
(IOException) java.io.IOException: unknown format (magic number 48)
(IOException) java.io.IOException: unknown format (magic number 1d)
(IOException) java.io.IOException: unknown format (magic number 3a)

Can you confirm which function on the .net side is ok so I can focus on the b4a code?
 
Upvote 0

Ohanian

Active Member
Licensed User
Longtime User
Got no result, any other suggestion or approach?

related posts on stackoverflow :

B4X:
http://stackoverflow.com/questions/6717165/how-can-i-zip-and-unzip-a-string-using-gzipoutputstream-that-is-compatible-with
 
Upvote 0

Ohanian

Active Member
Licensed User
Longtime User
Based on this link the .Net implementation adds 4 bytes (maybe with the length I haven't read it all).

Unless the text is very large, you will not gain anything from compressing it with gzip and then converting it back to a base64 string. For shorter texts the messages will be longer.

the text is a query result and its about 2mb , with compression it becomes 100kb!
i'll try another component (like xceed) to compress the string and decompress it in b4a.
 
Upvote 0

Ohanian

Active Member
Licensed User
Longtime User
Hi,

Here's a working solution :

.Net part :
B4X:
    Function GZipCompress(ByVal s As String) As String
        Dim bytes = Encoding.UTF8.GetBytes(s)
        Using msi = New MemoryStream(bytes)
            Using mso = New MemoryStream()
                Using gs = New GZipStream(mso, CompressionMode.Compress)
                    msi.CopyTo(gs)
                End Using
                Return Convert.ToBase64String(mso.ToArray())
            End Using
        End Using
    End Function

B4A part :
B4X:
#If Java

import android.Manifest;
import android.content.Context;
import android.os.Build;
import java.io.File;
import java.io.IOException;
import java.util.Collections;
import java.util.List;
import java.util.UUID;
import java.io.ByteArrayOutputStream;
import java.util.zip.GZIPOutputStream;
import java.io.ByteArrayInputStream;
import java.util.zip.GZIPInputStream;

    public byte[] compress(String string) {
        try {
            ByteArrayOutputStream os = new ByteArrayOutputStream(string.length());
            GZIPOutputStream gos = new GZIPOutputStream(os);
            gos.write(string.getBytes());
            gos.close();
            byte[] compressed = os.toByteArray();
            os.close();
            return compressed;
        } catch (IOException ex) {
            return null;
        }
    }

    public String decompress(byte[] compressed) {
        try {
            final int BUFFER_SIZE = 32;
            ByteArrayInputStream is = new ByteArrayInputStream(compressed);
            GZIPInputStream gis = new GZIPInputStream(is, BUFFER_SIZE);
            StringBuilder string = new StringBuilder();
            byte[] data = new byte[BUFFER_SIZE];
            int bytesRead;
            while ((bytesRead = gis.read(data)) != -1) {
                string.append(new String(data, 0, bytesRead));
            }
            gis.close();
            is.close();
            return string.toString();
        } catch (IOException ex) {
            return null;
        }
    }    
   
    // Base64
    public String compressBase64(String strToCompress) {
        byte[] compressed = compress(strToCompress);
        String encoded = android.util.Base64.encodeToString(compressed, android.util.Base64.NO_WRAP);
        return encoded;
    }
   
    public String decompressBase64(String strEncoded) {
        byte[] decoded = android.util.Base64.decode(strEncoded, android.util.Base64.NO_WRAP);
        String decompressed = decompress(decoded);
        return decompressed;
    }

#End If
 
Upvote 0
Top