Andrea Girardi - It's my blog!

Tag: MongoDB

Date based query using milliseconds (Java long) time on MongoDB

Let’s suppose I need to search all records that match a date condition. On MongoDB I’ve a bunch of data like this:

{
    "_id" : "9ed3b937-0f43-4613-bd58-cb739a8c5bf6",
    "userModels" : {
        "5080" : {
            "generated_date_timestamp" : NumberLong(1413382499442),
            "model_id" : 5080,
        },
    }
    "values" : {}
}

This is the query:

db.anonProfile.find({ 
   "userModels.5080.generated_date_timestamp" : { 
      "$gte" : ISODate("2013-10-01T00:00:00.000Z").getTime() 
   }
});
.getTime()

allows to translate ISODate into a NumberLong timestamp.

Query date based using milliseconds time on MongoDB

I need to search all records that match a date condition. On MongoDB I’ve a bunch of data like this:

{
    "_id" : "9ed3b937-0f43-4613-bd58-cb739a8c5bf6",
    "userModels" : {
        "5080" : {
            "generated_date_timestamp" : NumberLong(1413382499442),
            "model_id" : 5080,
        },
    }
    "values" : {}
}

With this query, is possible to do date / time based search:

db.anonProfile.find({ 
   "userModels.5080.generated_date_timestamp" : 
      { 
         "$gte" : ISODate("2013-10-01T00:00:00.000Z").getTime() 
      }
});

Results pagination with MongoDB and Java

To implement the MongoDB results pagination in Java, there are few parameters that need to be collected:

1) the order of results (ascending / descending)
2) the field used to order the results
3) the number of element and the page selected
4) the field and the value used to filter the results

As well as the results of query, the method needs to return the total number of elements. All returned elements will be saved in a HashMap.

HashMap<String, Object> resultMap = new HashMap<String, Object>();
 
Direction direction = Sort.DEFAULT_DIRECTION;
if (sortDirection > 0) {
	direction = Sort.Direction.ASC;
} else { 
	direction = Sort.Direction.DESC;
}
 
List

If a pagination is required, skip and limit are used

if (pageSize > 0) {
	query.skip((pageNum - 1 ) * pageSize);
	query.limit(pageSize);
}
 
if ( sortField != null && !sortField.equals("") ) {				
	query.with(new Sort(direction, sortField));
}
 
results = mongoTemplate.find(query, Object.class);

If a pagination is required, queryCounter (basically a version of query without pagination an limit) is used to calculate the total number of results. Of course, if pagination is not required, is possible to use directly the size of results.

if ( pageSize > 0 ) {
	resultMap.put("RESULT_SIZE", (int) mongoTemplate.count(queryCounter, Object.class));
} else {
	// If pagination is not required, the query is not re-executed
	resultMap.put("RESULT_SIZE", results.size());
}

mongoTemplate is a spring bean defined in this way on context configuration:

<bean id="mongoTemplate" class="org.springframework.data.mongodb.core.MongoTemplate"  c:mongo-ref="mongo" c:databaseName="${mongo.db.name}">
	<property name="writeConcern" value="SAFE" />
</bean>

MongoDB query with logical and condition in Java

Suppose you need to apply some filters to your MongoDB query, for example to extract some _ids that match a regex condition. This is the way to do that:

Query query;
query.addCriteria(Criteria.where("_id").in(IDs).and(query_field).regex(".*" + query_value + ".*", "i"));

In this example I used the Query (see here) and Criteria (see here) classes

And, this is the query you can use in mongoDB (Robomongo* or command line):

Query: { 
	"_id" : { "$in" : [ "ID1" , "ID2" , "ID3" ]} , 
	"detail.medicationBrandName" : { "$regex" : ".*x.*" , "$options" : "i"}}, 
	Fields: null, Sort: { "medicationGenericName" : -1}
}

*Robomongo: is a shell-centric cross-platform open source MongoDB management tool (i.e. Admin GUI). Robomongo embeds the same JavaScript engine that powers MongoDB’s mongo shell. You can download it here.

Spring-data: Cannot use a complex object as a key value

I was trying to figured out how to solve this issue. Basically I’m saving a user profile bean that contains multiple occurrences of other beans inside him. Something like date:

public class UserProfile {
 
	List<Class1> classes1;
	List<Class2> classes2;
 
	Integer int1;
	Map<String, Class3> classes3;
 
}

I was working fine until I changed a getter adding some business functionalities and surrounding all lines with a try / catch. Of course, I putted the exception message on the log using adding this definition inside the bean:

protected final Log logger = LogFactory.getLog(getClass());

Good point for me, everything worked fine until I updated the object in Mongo. Holy crap, I got the “Cannot use a complex object as a key value” exception and I spent two hours trying to change the beans (of course I did a lot of other changes after the fatal adding of business logic inside my bean). Exhausted, I contacted my colleague Max (@maxfarnea) and he told me, “when I got this error, I removed the log inside a bean”. Ipso facto, I removed the log entry from my bean, added a throws exception instead of a try / catch and tested! Well done @maxfarnea, it works!

I learned two lesson today: one, don’t put log into beans that need to be stored using spring-data and, two, if you are in pain, ask, talk, don’t be silly, don’t waste your time!!! Stackoverflow.com is a good point to start (and my blog either, of course) but never is more helpful than a quick chat with a colleague!

Route a message to MongoDB

The requirement is very simple. Route an XML message from rabbitMQ to MongoDB. MongoDB BSON as the data storage and network transfer format for “documents”. BSON is a binary-encoded serialization of JSON-like documents. So, the source message is in a XML format, after getting it from rabbitMQ is necessary to translate into a JSON format compatible with MongoDB.

To translate a message from XML to JSON is possible to use Marshalling function available in Camel extension XmlJson. The pom.xml file needs this new entry:

<dependency>
	<groupId>org.apache.camel</groupId>
	<artifactId>camel-xmljson</artifactId>
	<version>${org.camel.version}</version>
</dependency>

Of course, to send communicate with MongoDB another entry has to be added to pom.xml:

<dependency>
	<groupId>org.apache.camel</groupId>
	<artifactId>camel-mongodb</artifactId>
	<version>${org.camel.version}</version>
</dependency>

Now, is possible to change the route on camel ApplicationContext file:

<!-- Camel route -->    
<camelContext xmlns="http://camel.apache.org/schema/spring">
	<dataFormats>
		<xmljson id="xmljson"/>
	</dataFormats>
	<route> 
		<from uri="spring-amqp:TPDirect:TPQueue:TPRouting?type=direct&amp;autodelete=true&amp;durable=true" />
		<marshal ref="xmljson"/>
		<log message="From XML to Json: DONE!" />
		<convertBodyTo type="java.lang.String"/>
		<to uri="mongodb:myDb?database=flights&amp;collection=tickets&amp;operation=save" />
	</route>                                 
</camelContext>
 
 
<!-- Mongo DB -->
<bean id="myDb" class="com.mongodb.Mongo">
	<constructor-arg index="0" value="localhost"/>
</bean>

Bean myDB contains the information to reach MongoDB. It’s also possible to define it using the full url:

<bean id="myDb" class="com.mongodb.Mongo"> 
    <constructor-arg index="0"> 
        <bean class="com.mongodb.MongoURI"> 
            <constructor-arg index="0" value="mongodb://username:password@host:port/db" /> 
        </bean> 
    </constructor-arg> 
</bean>

To avoid this exception:

Caused by: No type converter available to convert from type: byte[] to the required type: org.apache.camel.component.mongodb.converters.MongoDbBasicConverters with value

The JSON translated message has to be converted into String.

That’s all!

Copyright © 2017 Andrea Girardi – It's my blog!

Theme by Anders NorenUp ↑