Wednesday, 9 November 2016

Talend - Out of Memory Error and Java Heap Space Error

The Out of Memory Error and Java Heap Space Error are two of the usual errors which occur in the Talend jobs handling a large volume of data. These errors can be avoided to an extent by following some design guidelines.

(1) Keep in mind that tMap is a heavy component. Minimize its use in your jobs.

  • Avoid tMap if you need just simple transformations like trimming the string values, replacing null numbers by zeroes, etc. In its place you can use tJavaRow component.
  • If you want to get only a small set of columns from a huge collection avoid using a tMap. For that you can use a lighter component- tFilterColumns
  • Similarly, to filter rows you can use tFilterRow instead of a tMap
(2) Use store on disk option whenever necessary.
          This option is available in tMap, tUniqRow, tSortRow, etc.
  • tMap
While using store on disk option in tMap the directory to store temporary data will be created automatically. This data will not be deleted or replaced on subsequent run(s) of the job. So it is advised to delete the temporary directory created using tFileDelete component from within the job. You can give that in On Subjob Ok of tPostJob component.

  • tUniqRow
In the case of tUniqRow the temporary directory should be created manually before the job run/or can be handled within the job. If the temporary directory is not available, the component tUniqRow will give out FileNotFoundException!

  • tSortRow
In the case of tSortRow the temporary directory will be created automatically


(3) The JVM arguments can be modified as and when needed
.
-Xms256M - initial memory size available to JVM is 256 MB
-Xmx1024M - maximum memory size available to JVM is 1024 MB


TALEND Installation Instructions

A)Pre-requisites before Talend Installation :

1)      Talend User needs to be created, in order to access the PROJECTS in REMOTE Repository.
For Local Repository, you can igore this.

2)       Need to have Java  JDK  V1.6 or higher

Oracle JDK Setup Details :

1)      Install Oracle JDK 1.6 or Higher. You will need the JDK, not the JRE to develop jobs using the Talend 6 studio. You can download and install this here:

Talend Installer automatically Checks for JDK version 1.6 or higher version. If no instance of JDK is found, the installer will shutdown.

2)Set up your JAVA_HOME environment Path Variable.

Define your JAVA_HOME evironment variable so that it points to the JDK directory.
For example, if the JDK path is C:\java\JDKx.x.x\bin, then you must set the JAVA_HOME environment variable to point to C:\Java\JDKx.x.x.

It is highly recommended that the full path to the server installation directory is as short as possible and does not contain any space character. If you already have a suitable JDK installed in a path with a space, you simply need to put quotes around the path when setting the values for the environment variable.

If you use Talend Installer, you also have to set the Path system variable.

3)Add the previously defined JAVA_HOME variable to the Path environment variable.

For Example  <PathVariable>%JAVA_HOME%\bin.


Note : You can add the Path variable and  create the JAVA_HOME environment variable in System Variables Section.



Thursday, 8 September 2016

STRING RELATED FUNCTIONS IN TALEND

CONTAINS & EQUALS :

row1.contains("ram")
row2.equals("Name")

LASTINDEXOF :
("tFileList_1_CURRENT_FILE")).lastIndexOf("."))

SUBSTRING:
Input  :  "ABCD".substring(0,2)
Result :   ABC

GETTING CURRENT FILEPATH  IN tFILELIST :
((String)globalMap.get("tFileList_1_CURRENT_FILEPATH"))

GETTING CURRENT FILENAME FROM  tFILELIST :
((String)globalMap.get("tFileList_1_CURRENT_FILE")).substring(0,((String)globalMap.get
("tFileList_1_CURRENT_FILE")).lastIndexOf("."))

replace() Example :
-------------------------

we can use this for replacing special characters in a string
String input_row="abcd:[],kdkkxyzkll:[],";


 

String abc="{\"NOTIFICATION_RESPONSE\":{"+input_row.replace(":[],",":\"\"")+"}}";
System.out.println(abc);
Output :  
-----------
{"NOTIFICATION_RESPONSE":{abcd:""kdkkxyzkll:""}}
Note :  Go for replace() if  replaceAll() is not working.
replaceAll() will not work for dollar ($),doublequotes("),comma(,) etc special characters, in such
cases you can go for replace().

Friday, 19 August 2016

TALEND/TAC ERROR : TAC, TALEND JOB CONDUCTOR shows the TALEND job in SENDING state when the job is executed...

When a talend job is deployed in TAC  it stores the job in multiple places as below

1)C:\TAC Installed Folder\repository
2)C:\TAC Installed Folder\generated_jobs\task_NUMBER
3)C:\TAC Installed Folder\TalendJobServersFiles\archiveJobs

We got to delete the task_NUMBER(NUMBER will be generated by TAC once you deploy the job) folder in the path C:\TAC Installed Foldergenerated_jobs\

Once you delete the task, the hanging mode or SENDING state or  DEPLOYING state will be released automatically.


Tuesday, 16 August 2016

7 Tricks To Learn Any Language In 7 Days (From The Superpolyglot Twins Who Did It)

What’s possible in a week? If you dedicated seven days to the achievement of one goal, how ambitious could you make this goal? These were the questions that the multilingual twins Matthew and Michael Youlden posed themselves when they determined to learn Turkish in one week. They would attempt to liberate themselves from the distractions and responsibilities of modern-day life in order to cram eight hours of study time into their daily routine. Here are the seven things that I learned by observing some of the world’s most capable language learners.

1. Get To Know Why

Lesson learned: Clearly define your goal at the very beginning and then plot a route towards this goal’s achievement.
The twins set themselves the challenge of learning a language in a week in order to stretch themselves, and then it was a question of choosing which language to learn. Turkish presented itself as a natural option; there are nigh on 300,000 Turkish speakers in Germany’s capital, and the areas of Kreuzberg and Neukölln are dotted with stores adorned with signs in Turkish. Truly understanding one’s environment in these neighbourhoods requires one to first understand Turkish.

2. Get Sticky

Lesson learned: Map and label your immediate environment in the new language from the very first moment. You’ll build and reinforce associations passively while going about your daily life.
The first operational step in the twins’ learning process was to decorate the entire apartment with sticky notes. This had an almost ceremonial touch to it as the twins delved into dictionaries and proceeded to label everything with its corresponding Turkish name. Within the space of about an hour it was impossible to carry out any menial task, be it making a coffee or flicking off a light switch, without first being presented with at least three different words related to this action.

3. Get A Partner

Lesson learned: There are few better motivations than a peer with the same goal. Whether you’re motivated by competition or a sense of mutual responsibility, the mere presence of a learning partner is likely to exert just the right amount of pressure to keep you on track.
The importance of the other twin’s presence became immediately apparent as Matthew and Michael delegated responsibilities for rooms to decorate with sticky notes. This simple task was augmented by continuous little tests that they would spring on one another, and the fact that they split up their day slightly differently and studied different topics meant that each twin became a source of knowledge for the other; the question how do you say that again? was met surprisingly often with an answer. The most extraordinary moment came towards the end of the week when the twins simply switched their everyday conversations to Turkish, asking one another if they wanted tea or coffee, were ready to cook dinner or when they were going to leave the house the next day.

4. Prepare Mini-Motivations

Lesson learned: You need landmarks on your route towards your goal. These landmarks can consist of small challenges - real life interactions in the language, for example - which force you to prepare areas of vocabulary to overcome them. The gratification that will come with their completion will serve to spur you on to ever greater heights.
Matthew and Michael had numerous micro-challenges throughout the week. On the first day they were visited by a Turkish friend who greeted them in Turkish and complimented them on how quickly they’d picked up their first words and phrases. They then learned the names of fruits and the numbers from one to a billion so that they could visit the Turkish market in Kreuzberg (although they refrained from purchasing nine hundred thousand kumquats). Displaying their haul after their first functional exchange in Turkish, they beamed with pride and a palpable sense of accomplishment before marching back home to study further.

5. Eat The Language

Lesson learned: Find a way to tie everything you do to learning. Surround yourself with the food, the music and the films, so that even in your downtime you can prime your mind towards the language and perhaps trigger further areas of interest and motivation.
On our second visit to the brothers’ apartment 24 hours into the week, we found them sampling dozens of different kinds of Turkish snacks. Like kids staring at the backs of cereal packs before heading to school, the nutritional information and various special offers and competitions on the packaging were analysed during snack breaks. There was no moment of complete removal from the language learning process during the eight hours that the twins had allotted to it. The intensity ebbed and flowed, but it never dissipated entirely.

6. Use What You Already Know

Lesson learned: The greater the depth of processing, the more likely the information will be remembered. Find pleasure in drawing parallels and making comparisons between the language(s) you already know and your new language.
One of the twins’ most common phrases was, “ah, that’s a bit like in … ?” They were constantly using their existing knowledge to support the ever-growing knowledge of Turkish. Not only did this spark some energetic exchanges regarding the etymology of various words, but it also ensured new words would never be forgotten once woven into their web of associations. Even if you are learning your second language, you will likely come across words that share common origins with words in your native tongue.

7. Variation is the spice of life

Lesson learned: So you have your route plotted and an idea of your favored methods, but do remember to try new things; your new language has just as many sources as your native language.
The twins spent a lot of time engrossed in books or on their computers and apps, flicking and swiping their way through exercises eagerly, but at other times they were to be found searching busily for Turkish radio stations and write-ups of Turkish football games on the web. There is no definitive method to learn a language, nor any tool or teacher that will single-handedly deliver you to the holy grail of fluency. Language is written, spoken, read and heard. Each of these areas is considered a core skill within which there are myriad potential inputs; would you restrict yourself to one in your native language? All too often, people enter their weekly language class to converse with their teacher, but then barely have any contact with other native speakers or the media being broadcast in their target language. Try something new every day. Listen to a cheesy song, read a newspaper article from a newspaper whose politics differ from your own, write a story for kids, attempt some improvised theatre and talk to yourself while cooking. Spice it up and add some flavour to your language learning!

FINDING ACTIVE PORTS in WINDOWS

netstat -an


Are You Thinking?????????????????????

                      Question Yourself for few Seconds?                         Are you Thinking at all?                              When ...