Andrea Girardi - It's my blog!

Tag: java (page 1 of 3)

How to use Spring DataSource bean as data source for Log4j 2 JDBC appender

I would like to log log4j2 messages into a relational database using the datasource defined on application context and initialized using spring using log4j 2.10.

One possibility is to add a JDBC appender inside log4j2 xml configuration but, Log4j is initialized before Spring so, dataSource won’t be available at runtime so the only solution is to add an appender programmatically. Of course, it is possible to use a log4j2 jdbc appender but, this approach allow to have the possibility to use spring.properties file to override the spring application context with proper environment settings in according to my profile.

This is the datasource defined on application context xml:

<!--  ############ SQLSERVER DATABASE SECTION ############ -->
<bean id="dataSourceMSSqlServer" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
    <property name="driverClassName" value="com.microsoft.sqlserver.jdbc.SQLServerDriver" />
    <property name="url" value="jdbc:sqlserver://${sqlserver.hostname};databaseName=${sqlserver.database};" />
    <property name="username" value="${sqlserver.user}" />
    <property name="password" value="${sqlserver.pass}" />
</bean>

this allow me to configure different database for every environment.
This is the table on which I want to log the entries:

[Id] [INT] IDENTITY(1,1) NOT NULL,
[CreatedTimeStamp] [datetimeoffset](7) NOT NULL,
[Level] [INT] NOT NULL,
[SOURCE] [nvarchar](MAX) NULL,
[Message] [nvarchar](MAX) NULL,
[Content] [nvarchar](MAX) NULL,
[ProductName] [nvarchar](MAX) NULL,
[Version] [nvarchar](MAX) NULL,
[LogType] [INT] NOT NULL DEFAULT ((0)),
[AuditEventType] [INT] NULL,
[UserId] [nvarchar](128) NULL,

The plan is to create a spring bean, inject the DataSource bean, and add JDBC Appender configuration dynamically in @PostConstruct method.

package com.afm.web.utility;
 
import java.sql.Connection;
import java.sql.SQLException;
 
import javax.annotation.PostConstruct;
import javax.sql.DataSource;
 
import org.apache.logging.log4j.Level;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.core.Appender;
import org.apache.logging.log4j.core.LoggerContext;
import org.apache.logging.log4j.core.appender.db.ColumnMapping;
import org.apache.logging.log4j.core.appender.db.jdbc.ColumnConfig;
import org.apache.logging.log4j.core.appender.db.jdbc.ConnectionSource;
import org.apache.logging.log4j.core.appender.db.jdbc.JdbcAppender;
import org.apache.logging.log4j.core.config.AppenderRef;
import org.apache.logging.log4j.core.config.Configuration;
import org.apache.logging.log4j.core.config.LoggerConfig;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
 
@Component
public class JDBCLog {
 
    @Autowired
    private DataSource dataSourceMSSqlServer;
 
    // Inner class
    class Connect implements ConnectionSource {
 
	private DataSource dsource;
 
	public Connect(DataSource dsource) {
	    this.dsource = dsource;
	}
 
	@Override
	public Connection getConnection() throws SQLException {
	    return this.dsource.getConnection();
	}
 
    }
 
    public JDBCLog() {}
 
    @PostConstruct
    private void init(){
 
	System.out.println("####### JDBCLog init() ########");      
	final LoggerContext ctx = (LoggerContext) LogManager.getContext(false); 
	final Configuration config = ctx.getConfiguration();
 
	// Here I define the columns I want to log. 
	ColumnConfig[] columnConfigs = new ColumnConfig[] {
	    ColumnConfig.newBuilder()
                .setName("CreatedTimeStamp")
                .setPattern(null)
                .setLiteral(null)
                .setEventTimestamp(true)
                .setUnicode(false)
                .setClob(false).build(),
	    ColumnConfig.newBuilder()
                .setName("Source")
                .setPattern("%K{className}")
                .setLiteral(null)
                .setEventTimestamp(false)
                .setUnicode(false)
                .setClob(false).build(),
	    ColumnConfig.newBuilder()
                .setName("Level")
                .setPattern("%level")
                .setLiteral(null)
                .setEventTimestamp(false)
                .setUnicode(false)
                .setClob(false).build(),
	    ColumnConfig.newBuilder()
                .setName("Message")
                .setPattern("%K{message}")
                .setLiteral(null)
                .setEventTimestamp(false)
                .setUnicode(false)
                .setClob(false).build(),
	    ColumnConfig.newBuilder()
                .setName("Content")
                .setPattern("%K{exception}")
                .setLiteral(null)
                .setEventTimestamp(false)
                .setUnicode(false)
                .setClob(false).build(),
	    ColumnConfig.newBuilder()
                .setName("ProductName")
                .setPattern(null)
                .setLiteral("'DHC'")
                .setEventTimestamp(false)
                .setUnicode(false)
                .setClob(false).build(),
	    ColumnConfig.newBuilder()
                .setName("Version")
                .setPattern(null)
                .setLiteral("'1.0'")
                .setEventTimestamp(false)
                .setUnicode(false)
                .setClob(false).build(),
	    ColumnConfig.newBuilder()
                .setName("AuditEventType")
                .setPattern("%K{eventId}")
                .setLiteral(null)
                .setEventTimestamp(false)
                .setUnicode(false)
                .setClob(false).build(),
	    ColumnConfig.newBuilder()
                .setName("UserId"
                .setPattern("%K{userId}")
                .setLiteral(null)
                .setEventTimestamp(false)
                .setUnicode(false)
                .setClob(false).build(),
	    ColumnConfig.newBuilder()
                .setName("LogType")
                .setPattern("%K{logType}")
                .setLiteral(null)
                .setEventTimestamp(false)
                .setUnicode(false)
                .setClob(false).build()
        };
 
	Appender jdbcAppender = JdbcAppender.newBuilder()
		.setBufferSize(0)
                .setColumnConfigs(columnConfigs)
                .setColumnMappings(new ColumnMapping[]{})
                .setConnectionSource(new Connect(dataSourceMSSqlServer))
                .setTableName("dhc.LogItems")
                .withName("databaseAppender")
                .withIgnoreExceptions(true)
                .withFilter(null)
                .build();
 
	jdbcAppender.start();
	config.addAppender(jdbcAppender);
 
	// Create an Appender reference.
	// @param ref The name of the Appender.
	// @param level The Level to filter against.
	// @param filter The filter(s) to use.
	// @return The name of the Appender.
	AppenderRef ref= AppenderRef.createAppenderRef("JDBC_Appender", null, null);
        AppenderRef[] refs = new AppenderRef[] {ref};
 
        /*
         * Factory method to create a LoggerConfig.
         *
         * @param additivity true if additive, false otherwise.
         * @param level The Level to be associated with the Logger.
         * @param loggerName The name of the Logger.
         * @param includeLocation whether location should be passed downstream
         * @param refs An array of Appender names.
         * @param properties Properties to pass to the Logger.
         * @param config The Configuration.
         * @param filter A Filter.
         * @return A new LoggerConfig.
         * @since 2.6
         */
        LoggerConfig loggerConfig = LoggerConfig.createLogger(
                false, Level.DEBUG, "JDBC_Logger", null, refs, null, config, null);        
        loggerConfig.addAppender(jdbcAppender, null, null);
 
        config.addLogger("JDBC_Logger", loggerConfig);       
        ctx.updateLoggers();  
 
        System.out.println("####### JDBCLog init() - DONE ########");  
 
    }
 
    public DataSource getDataSource() {
	return dataSourceMSSqlServer;
    }
 
    public void setDataSource(DataSource dataSourceMSSqlServer) {
	this.dataSourceMSSqlServer = dataSourceMSSqlServer;
    }	  
 
}

At this point, is possible to call the logger from the code in this way:

Logger jdbcLogger = LogManager.getContext(false).getLogger("JDBC_Logger"); 
jdbcLogger.info(new StringMapMessage()
    .with("eventId", AuditEventType.Logger_General.toString())
    .with("exception", "")
    .with("userId", "TESTUSER")
    .with("message", "TEST!!")
    .with("className", 
        this.getClass().getPackage().toString().replaceAll("package ", "") 
        + "." + this.getClass().getSimpleName() 
        + "." + new Object() {}.getClass().getEnclosingMethod().getName())
);

If you are using Log4j in a Servlet 2.5 web application, or if you have disabled auto-initialization with the isLog4jAutoInitializationDisabled context parameter, you must configure the Log4jServletContextListener and Log4jServletFilter in the deployment descriptor or programmatically. The filter should match all requests of any type. The listener should be the very first listener defined in your application, and the filter should be the very first filter defined and mapped in your application. This is easily accomplished using the following web.xml code:

<listener>
    <listener-class>
        org.apache.logging.log4j.web.Log4jServletContextListener
    </listener-class>
</listener>
 
<filter>
    <filter-name>log4jServletFilter</filter-name>
    <filter-class>
        org.apache.logging.log4j.web.Log4jServletFilter
    </filter-class>
</filter>
<filter-mapping>
    <filter-name>log4jServletFilter</filter-name>
    <url-pattern>/*</url-pattern>
    <dispatcher>REQUEST</dispatcher>
    <dispatcher>FORWARD</dispatcher>
    <dispatcher>INCLUDE</dispatcher>
    <dispatcher>ERROR</dispatcher>
    <!-- 
        Servlet 3.0 w/ disabled auto-initialization only; 
        not supported in 2.5 
    -->
    <dispatcher>ASYNC</dispatcher>
</filter-mapping>

This dependency must be added to pom.xml:

<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-web</artifactId>
    <version>2.10.0</version>
</dependency>

Of course, if you are not already included, add it to the component-scan into Spring application-context:

<context:component-scan base-package="com.afm.web.utility" />

PS: if you like it please, click on the banner :)

Warning about SSL connection when connecting to MySQL database

After a recent update of mySql, I get this warning:

WARN: Establishing SSL connection without server's identity verification is not recommended. 
According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established 
by default if explicit option isn't set. For compliance with existing applications not using SSL 
the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL 
by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.

To disable disable SSL and also suppress the SSL warning, it’s possible to set to false the useSSL parameter on the connection string:

jdbc:mysql://localhost:3306/myDb?autoReconnect=true&useSSL=false

On applicationContext, something like this:

<!--  ############ MY SQL DATABASE SECTION ############ -->
<bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">	
    <property name="driverClassName"><value>com.mysql.jdbc.Driver</value></property>
    <property name="url">
        <value>jdbc:mysql://${mysql.hostname}:${mysql.port}/${mysql.db}?useSSL=false</value>
    </property>
    <property name="username"><value>${mysql.user}</value></property>
    <property name="password"><value>${mysql.password}</value></property>
</bean>

Convert timestamp long to normal date format

One simply way to convert a Long time stamp into a formatted string is (time paramter is Long timestamp):

Date date = new Date(time);
Format format = new SimpleDateFormat("yyyy MM dd HH:mm:ss");
return format.format(date);

These packages must be included.

import java.sql.Date;
import java.text.Format;
import java.text.SimpleDateFormat;

Date and Time Patterns

Date and time formats are specified by date and time pattern strings. Within date and time pattern strings, unquoted letters from 'A' to 'Z' and from 'a' to 'z' are interpreted as pattern letters representing the components of a date or time string. Text can be quoted using single quotes (') to avoid interpretation. "''" represents a single quote.
All other characters are not interpreted; they’re simply copied into the output string during formatting or matched against the input string during parsing.

The following pattern letters are defined (all other characters from 'A' to 'Z' and from 'a' to 'z' are reserved):

Letter Date or Time Component Presentation Examples
G Era designator Text AD
y Year Year 1996; 96
M Month in year Month July; Jul; 07
w Week in year Number 27
W Week in month Number 2
D Day in year Number 189
d Day in month Number 10
F Day of week in month Number 2
E Day in week Text Tuesday; Tue
a Am/pm marker Text PM
H Hour in day (0-23) Number 0
k Hour in day (1-24) Number 24
K Hour in am/pm (0-11) Number 0
h Hour in am/pm (1-12) Number 12
m Minute in hour Number 30
s Second in minute Number 55
S Millisecond Number 978
z Time zone General time zone Pacific Standard Time; PST; GMT-08:00
Z Time zone RFC 822 time zone -0800

How do I fix a 65535 bytes limit Stacktrace?

It is possible that after an upgrade you may encounter this error on some of the more complex pages and root cause provided by tomcat console is :

Unable to compile class for JSP
The code of method _jspService(HttpServletRequest, HttpServletResponse) is exceeding the 65535 bytes limit

To solve the issue you need to locate the file [Tomcat_Home]/conf/web.xml and search the file for ‘JspServlet’. This should return an xml node of containing some values. You will need to add an additional the same as the below.

<init-param>
     <param-name>mappedfile</param-name>
     <param-value>false</param-value>
</init-param>

The resulting block of the web.xml file, once you have inserted the above, should look like the code below.

<servlet>
	<servlet-name>jsp</servlet-name>
	<servlet-class>org.apache.jasper.servlet.JspServlet</servlet-class>
	<init-param>
	    <param-name>fork</param-name>
	    <param-value>false</param-value>
	</init-param>
	<init-param>
	    <param-name>xpoweredBy</param-name>
	    <param-value>false</param-value>
	</init-param>
	<init-param>
	     <param-name>mappedfile</param-name>
	     <param-value>false</param-value>
	</init-param>         
	<load-on-startup>3</load-on-startup>
</servlet>

Save the file and restart the Tomcat service.

MappingJacksonHttpMessageConverter not found

I was trying to run a java batch that call an application context without success (it’s a java app that calls a Camel spring context). This is what I get during the startup:

ERROR ApplicationProperties @ addApplicationProperty [28] org.springframework.beans.factory.BeanDefinitionStoreException: Unexpected exception parsing XML document from file [/MessageRouting/src/test/resources/META-INF/spring/LOCALHOST-db-context.xml]; nested exception is java.lang.NoClassDefFoundError: org/springframework/http/converter/json/MappingJacksonHttpMessageConverter
Fatal error! java.lang.RuntimeException: Error loading ClassPathXmlApplicationContext file - src/test/resources/META-INF/spring/LOCALHOST-db-context.xml

I’m using spring 4.2.3 (updated yesterday, probably the reason for which it doesn’t work. It was 4.0.9). I know that MappingJacksonHttpMessageConverter has been replaced by MappingJackson2HttpMessageConverter but, how can I tell to spring to use the new version???

I followed some suggestion and I added this to my pom.xml:

<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-core</artifactId>
    <version>2.6.3</version>
</dependency>
<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>2.6.3</version>
</dependency>   
<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-annotations</artifactId>
    <version>2.6.3</version>
</dependency>

But this didn’t fix so, I finally resolved the issue adding this dependency to my pom.xml:

<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-webmvc</artifactId>
    <version>${org.springframework-version}</version>
</dependency>

Which log4j configuration are you using?

Sometimes log4j is not working properly and you need to verify where it is actually writing the log entries. This can be easily verified turning on the log4j.debug:

-Dlog4j.debug

It will print to System.out lots of helpful information about which file it used to initialize itself, which loggers / appenders got configured and how etc.

Parse an unknown JSON with Jquery

Sometime it happens to receive a JSON string than need to be visualized without knowing the structure. Suppose we have an HTML table:

<tbody id="reportTable">					
</tbody>

To populate this table with jQuery it’s possible to use this simple code:

var rows = ${reportRows};
var html = $.each(rows, function(key, value){
 
	$("#reportTable").append("<tr>");
 
	$.each(value, function (key, data) {
		$("#reportTable").append("<td>" + data + "</td>");
	});
 
	$("#reportTable").append("</tr>");
 
});

${reportRows} comes from a Spring MVC Controller and it’s a Java string generates using Jackson (or Gson) in this way:

ColumnMapRowMapper rowMapper = new ColumnMapRowMapper();
List<Map<String, Object>> reportDataList =  getJdbcTemplate().query(sqlComplete, rowMapper);
 
ObjectWriter ow = new ObjectMapper().writer().withDefaultPrettyPrinter();
String json = ow.writeValueAsString(reportDataList);

So, we have a query that return a not know number of column (suppose your code dynamically generate the query), we translate the results in JSON and we use jQuery to render the results.

Dynamic for with Spring MVC using a HashMap

Sometime you need to dynamically generate a form without knowing how many fields it will be required (i.e. when your form is driven by a configuration or by some properties). The problem is to draw the form and, return the values to the Controller, and recognize the couples Field name / Field value after the submit..

You can easily get solve this problem just adding HashMap which will hold the key-value pair data to the DataModel.

Supposing your configuration says: you have to draw two fields and these are the name, you UI will be something like*:

<c:forEach items="${newRequest.fields}" var="field">
	<f:input type="text" path="rawFields['${field.field_id}']" class="form-control validate[groupRequired[mandatoryField]]" /> (R)
</c:forEach>

When you submit the form, the values and the key for the dynamic fields will be filled.

* newRequest is the DataModel you are passing and fields is the list of Fields that user will fill with data, like that:

public class Request {
 
	/** Request type */
	private int templateRequest;
 
	// ***** Input field ***** 
	List<RequestField> fields = new ArrayList<RequestField>();
 
	private HashMap<String, Object> rawFields = new HashMap<String, Object>();
 
	[Setters and getters]
 
}

Results pagination with MongoDB and Java

To implement the MongoDB results pagination in Java, there are few parameters that need to be collected:

1) the order of results (ascending / descending)
2) the field used to order the results
3) the number of element and the page selected
4) the field and the value used to filter the results

As well as the results of query, the method needs to return the total number of elements. All returned elements will be saved in a HashMap.

HashMap&lt;String, Object&gt; resultMap = new HashMap&lt;String, Object&gt;();
 
Direction direction = Sort.DEFAULT_DIRECTION;
if (sortDirection &gt; 0) {
	direction = Sort.Direction.ASC;
} else { 
	direction = Sort.Direction.DESC;
}
 
List

If a pagination is required, skip and limit are used

if (pageSize &gt; 0) {
	query.skip((pageNum - 1 ) * pageSize);
	query.limit(pageSize);
}
 
if ( sortField != null &amp;&amp; !sortField.equals("") ) {				
	query.with(new Sort(direction, sortField));
}
 
results = mongoTemplate.find(query, Object.class);

If a pagination is required, queryCounter (basically a version of query without pagination an limit) is used to calculate the total number of results. Of course, if pagination is not required, is possible to use directly the size of results.

if ( pageSize &gt; 0 ) {
	resultMap.put("RESULT_SIZE", (int) mongoTemplate.count(queryCounter, Object.class));
} else {
	// If pagination is not required, the query is not re-executed
	resultMap.put("RESULT_SIZE", results.size());
}

mongoTemplate is a spring bean defined in this way on context configuration:

<bean id="mongoTemplate" class="org.springframework.data.mongodb.core.MongoTemplate"  c:mongo-ref="mongo" c:databaseName="${mongo.db.name}">
	<property name="writeConcern" value="SAFE" />
</bean>

MongoDB query with logical and condition in Java

Suppose you need to apply some filters to your MongoDB query, for example to extract some _ids that match a regex condition. This is the way to do that:

Query query;
query.addCriteria(Criteria.where("_id").in(IDs).and(query_field).regex(".*" + query_value + ".*", "i"));

In this example I used the Query (see here) and Criteria (see here) classes

And, this is the query you can use in mongoDB (Robomongo* or command line):

Query: { 
	"_id" : { "$in" : [ "ID1" , "ID2" , "ID3" ]} , 
	"detail.medicationBrandName" : { "$regex" : ".*x.*" , "$options" : "i"}}, 
	Fields: null, Sort: { "medicationGenericName" : -1}
}

*Robomongo: is a shell-centric cross-platform open source MongoDB management tool (i.e. Admin GUI). Robomongo embeds the same JavaScript engine that powers MongoDB’s mongo shell. You can download it here.

Olderposts

Copyright © 2018 Andrea Girardi – It's my blog!

Theme by Anders NorenUp ↑