ConvNet  1.0
A GPU-based C++ implementation of Convolutional Neural Nets
 All Classes Namespaces Functions Variables
Public Member Functions | Protected Member Functions | Protected Attributes | List of all members
DataHandler Class Reference

Makes data accessible to the model. More...

#include <datahandler.h>

Public Member Functions

 DataHandler (const config::DatasetConfig &config)
 
void GetBatch (vector< Layer * > &data_layers)
 
int GetBatchSize () const
 
int GetDataSetSize () const
 
void Seek (int row)
 
void Preprocess (Matrix &input, Matrix &output)
 
void Sync ()
 

Protected Member Functions

void SetupShuffler ()
 
void ShuffleIndices ()
 
void LoadChunk (DataIterator &it, Matrix &mat)
 
void LoadChunk (DataIterator &it, Matrix &mat, vector< int > &random_rows)
 
void PipelinedDiskAccess ()
 
void DiskAccess ()
 
void StartPreload ()
 
void WaitForPreload ()
 

Protected Attributes

default_random_engine generator_
 
uniform_int_distribution< int > * distribution_
 
map< string, DataIterator * > data_it_
 
map< string, Matrixdata_
 
vector< string > layer_names_
 
thread * preload_thread_
 
Matrix rand_perm_indices_
 
int batch_size_
 
int chunk_size_
 
int max_reuse_count_
 
int reuse_counter_
 
int random_access_chunk_size_
 
int dataset_size_
 
int start_
 
bool restart_
 
bool nothing_on_gpu_
 
bool fits_on_gpu_
 
const bool pipeline_loads_
 
const bool randomize_cpu_
 
const bool randomize_gpu_
 

Detailed Description

Makes data accessible to the model.

Provides a GetBatch() method that is used by the model to fetch mini-batches. Handles multiple streams of data.


The documentation for this class was generated from the following files: