• Home
  • About

Snippet IT

IT News, Programming, Internet and Blogging

  • Programming and Scripting
  • Tips and Tricks
  • Software and Hardware
  • New and Happening
You are here: Home / Programming and Scripting / Java: Continuously Read Data From FileChannel Without MappedByteBuffer

Java: Continuously Read Data From FileChannel Without MappedByteBuffer

December 17, 2009 by Sze Hau 1 Comment

When programmer writes a program to read data from a file continuously and sequentially with a fixed data type reading sequence (e.g. to read three integers then 2 doubles repeatedly), he or she may find the MappedByteBuffer from Java library is useful.

MappedByteBuffer is pretty useful because its content is a memory-mapped region of a file. Program can access to the position of the file it wants directly. It also does some optimization on loading data from file, where part of the content are loaded into memory before the read action is actually performed. But when the data will be load? It remains unknown and it is operating system dependent. This is the first problem when using MappedByteBuffer – are the data buffered?

The second problem arises when programmer uses MappedByteBuffer to read a big file. MappedByteBuffer only allows data reading up to the maximum value of an integer (2,147,483,648 bytes). If the program needs to read further from that, it will need to re-map the file by calling the FileChannle.map function by parsing in the correct file position. The programmer will need to be very careful in passing the correct file position because some time it can be very confusing.

To overcome these two problems, we can introduce, design and write a new class that ensures the data reading action is buffered for the best performance and also allows continuous of reading without the need of re-mapping the FileChannle. First, let’s write out the pseudo code for the read file action:

[java]
// get data (e.g. get integer)
// ensure the memory buffer has enough data for the requested data type
// if not enough, read data from file
// read the data from buffer and return them in the requested data type
[/java]

Below is the example of implementation:
[java]
public class FileChannelReader {

private final static int BUFFER_SIZE = 64 * 1024; // 64k

private ByteBuffer readerBuffer;
private FileChannel readerFileChannel;

public FileChannelReader(FileChannel fileChannel) {
readerBuffer = ByteBuffer.allocate(PAGE_SIZE);
readerFileChannel = fileChannel;
readerBuffer.clear();
readerBuffer.flip();
}

// Ensure the buffer has enough data
private void ensureData(int size) throws IOException {
if (readerBuffer.remaining() < size) {
readerBuffer.compact();
if (readerFileChannel.read(readerBuffer) <= 0)
throw new IOException("Unexpected end-of-stream");
readerBuffer.flip();
}
}

// Get current position
public long position() throws IOException {
return readerFileChannel.position() – readerBuffer.remaining();
}

// Set current position
public void position(long position) throws IOException {
readerFileChannel.position(position);
readerBuffer.clear();
readerBuffer.flip();
}

// Get integer
public int getInt() throws IOException {
ensureData(Integer.SIZE/8);
return readerBuffer.getInt();
}

// Get long
public long getLong() throws IOException {
ensureData(Long.SIZE/8);
return readerBuffer.getLong();
}

// And so on …
}
[/java]

I’m using this technique in my programming projects and it performs pretty well. The data to be read is always ensured to be buffered and it also reduces complexity in writing a program to read data sequentially.

More from my site

  • Java: How To Select Top N Objects From A ListJava: How To Select Top N Objects From A List
  • Java: How To Create A Simple Web Server Using HttpServerJava: How To Create A Simple Web Server Using HttpServer
  • Java: Format Integer Into Fixed Width StringJava: Format Integer Into Fixed Width String
  • Java: Loading Large Data into JTable or JListJava: Loading Large Data into JTable or JList
  • Java: Format Long Integer Into Hexadecimal StringJava: Format Long Integer Into Hexadecimal String
  • Java: Stop A Thread CorrectlyJava: Stop A Thread Correctly

Filed Under: Programming and Scripting, Tips and Tricks Tagged With: FileChannel, how to, Java, MappedByteBuffer

About Sze Hau

Geek. Love programming. Coffee addicted. Married with two children. Working towards financial freedom.

Trackbacks

  1. Filechannel clear | Laughtogethert says:
    March 13, 2012 at 3:30 pm

    […] Java: Continuously Read Data From FileChannel Without …Java: Continuously Read Data From FileChannel Without MappedByteBuffer … readerFileChannel = fileChannel; readerBuffer.clear(); readerBuffer.flip(); … […]

Leave a Reply Cancel reply

Advertisement

  • Facebook
  • Google+
  • Instagram
  • Twitter

Email News Letter

Sign up to receive updates daily and to hear what's going on with us

Software and Hardware

MD5 and SHA1 Checksum Using Windows

July 5, 2017 By Sze Hau Leave a Comment

Blog Network

  • Personal Fincance Personal Finance – Personal Money Tips, Stock Investment, Small Business and Make Money Online
  • szehau's weblog Life, Internet, Software, Gadgets, Programming and Investments

Snippet IT

This is the place where I want to share anything about information technology.

Search

Recent

  • MD5 and SHA1 Checksum Using Windows
  • MD5 and SHA1 Checksum Using Linux
  • Java: Unlimited Strength Jurisdiction Policy
  • WordPress: How To Change Admin Username
  • Linux: How To Compress And Decompress Folders And Files

Tags

Adsense advertisement advertising apache blog blogging tips C# EGPC error estimation format format Integer Gmail Google Google Adsense Google Chrome Google Search Engine Google search result how to HTTP internet marketing Java JavaScript Linux money password performance PHP programming search engine optimization secure security short URL SQL static constructor String tiny URL Tips and Tricks twitter video Windows Vista Wordpress wordpress plugin wordpress theme Youtube

Copyright © 2023 · Magazine Pro Theme on Genesis Framework · WordPress · Log in