{"id":165,"date":"2013-02-24T20:01:00","date_gmt":"2013-02-24T19:01:00","guid":{"rendered":"https:\/\/nikostotz.de\/blog\/?p=165"},"modified":"2013-02-24T20:01:00","modified_gmt":"2013-02-24T19:01:00","slug":"merge-text-files-with-ant","status":"publish","type":"post","link":"https:\/\/www.nikostotz.de\/blog\/merge-text-files-with-ant\/","title":{"rendered":"Merge Text Files with Ant"},"content":{"rendered":"<p>\n<a href=\"https:\/\/ant.apache.org\/\">Apache Ant<\/a> has lots of useful built-in <a title=\"Ant Tasks\" href=\"https:\/\/ant.apache.org\/manual\/tasksoverview.html\">tasks<\/a>. However, a task for merging text files is not among them. Fortunately, we can combine some built-in tasks to achieve this goal.\n<\/p>\n<p>\nBelow, you&#8217;ll find the macro along a simple demo:<\/p>\n<div class=\"codecolorer-container xml default\" style=\"overflow:auto;white-space:nowrap;height:300px;\"><table cellspacing=\"0\" cellpadding=\"0\"><tbody><tr><td class=\"line-numbers\"><div>1<br \/>2<br \/>3<br \/>4<br \/>5<br \/>6<br \/>7<br \/>8<br \/>9<br \/>10<br \/>11<br \/>12<br \/>13<br \/>14<br \/>15<br \/>16<br \/>17<br \/>18<br \/>19<br \/>20<br \/>21<br \/>22<br \/>23<br \/>24<br \/>25<br \/>26<br \/>27<br \/>28<br \/>29<br \/>30<br \/>31<br \/>32<br \/>33<br \/>34<br \/>35<br \/>36<br \/><\/div><\/td><td><div class=\"xml codecolorer\"><span class=\"sc3\"><span class=\"re1\">&lt;project<\/span> <span class=\"re0\">default<\/span>=<span class=\"st0\">&quot;merge-demo&quot;<\/span><span class=\"re2\">&gt;<\/span><\/span><br \/>\n<br \/>\n<span class=\"sc3\"><span class=\"re1\">&lt;target<\/span> <span class=\"re0\">name<\/span>=<span class=\"st0\">&quot;merge-demo&quot;<\/span><span class=\"re2\">&gt;<\/span><\/span><br \/>\n&nbsp; &nbsp; <span class=\"sc3\"><span class=\"re1\">&lt;merge<\/span> <span class=\"re0\">tofile<\/span>=<span class=\"st0\">&quot;merged.txt&quot;<\/span><span class=\"re2\">&gt;<\/span><\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; <span class=\"sc3\"><span class=\"re1\">&lt;sourcefiles<span class=\"re2\">&gt;<\/span><\/span><\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"sc3\"><span class=\"re1\">&lt;fileset<\/span> <span class=\"re0\">dir<\/span>=<span class=\"st0\">&quot;inputfiles&quot;<\/span><span class=\"re2\">&gt;<\/span><span class=\"re1\">&lt;\/fileset<span class=\"re2\">&gt;<\/span><\/span><\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; <span class=\"sc3\"><span class=\"re1\">&lt;\/sourcefiles<span class=\"re2\">&gt;<\/span><\/span><\/span><br \/>\n&nbsp; &nbsp; <span class=\"sc3\"><span class=\"re1\">&lt;\/merge<span class=\"re2\">&gt;<\/span><\/span><\/span><br \/>\n<span class=\"sc3\"><span class=\"re1\">&lt;\/target<span class=\"re2\">&gt;<\/span><\/span><\/span><br \/>\n<br \/>\n&nbsp; &nbsp; <span class=\"sc-1\">&lt;!-- = = = = = = = = = = = = = = = = =<\/span><br \/>\n<span class=\"sc-1\"> &nbsp; &nbsp; &nbsp;macrodef: merge &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;<\/span><br \/>\n<span class=\"sc-1\"> &nbsp; &nbsp; = = = = = = = = = = = = = = = = = --&gt;<\/span><br \/>\n&nbsp; &nbsp; <span class=\"sc3\"><span class=\"re1\">&lt;macrodef<\/span> <span class=\"re0\">name<\/span>=<span class=\"st0\">&quot;merge&quot;<\/span><span class=\"re2\">&gt;<\/span><\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; <span class=\"sc3\"><span class=\"re1\">&lt;attribute<\/span> <span class=\"re0\">name<\/span>=<span class=\"st0\">&quot;tofile&quot;<\/span><span class=\"re2\">&gt;<\/span><span class=\"re1\">&lt;\/attribute<span class=\"re2\">&gt;<\/span><\/span><\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; <span class=\"sc3\"><span class=\"re1\">&lt;element<\/span> <span class=\"re0\">name<\/span>=<span class=\"st0\">&quot;sourcefiles&quot;<\/span><span class=\"re2\">&gt;<\/span><span class=\"re1\">&lt;\/element<span class=\"re2\">&gt;<\/span><\/span><\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; <span class=\"sc3\"><span class=\"re1\">&lt;sequential<span class=\"re2\">&gt;<\/span><\/span><\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"sc3\"><span class=\"re1\">&lt;tempfile<\/span> <span class=\"re0\">property<\/span>=<span class=\"st0\">&quot;temp.file&quot;<\/span> <span class=\"re0\">prefix<\/span>=<span class=\"st0\">&quot;ant.merge.&quot;<\/span> <span class=\"re0\">deleteonexit<\/span>=<span class=\"st0\">&quot;true&quot;<\/span><span class=\"re2\">&gt;<\/span><span class=\"re1\">&lt;\/tempfile<span class=\"re2\">&gt;<\/span><\/span><\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"sc3\"><span class=\"re1\">&lt;concat<\/span> <span class=\"re0\">destfile<\/span>=<span class=\"st0\">&quot;${temp.file}&quot;<\/span> <span class=\"re0\">fixlastline<\/span>=<span class=\"st0\">&quot;true&quot;<\/span><span class=\"re2\">&gt;<\/span><\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"sc3\"><span class=\"re1\">&lt;sourcefiles<span class=\"re2\">&gt;<\/span><\/span><span class=\"re1\">&lt;\/sourcefiles<span class=\"re2\">&gt;<\/span><\/span><\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"sc3\"><span class=\"re1\">&lt;\/concat<span class=\"re2\">&gt;<\/span><\/span><\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"sc3\"><span class=\"re1\">&lt;copy<\/span> <span class=\"re0\">file<\/span>=<span class=\"st0\">&quot;${temp.file}&quot;<\/span> <span class=\"re0\">tofile<\/span>=<span class=\"st0\">&quot;@{tofile}&quot;<\/span><span class=\"re2\">&gt;<\/span><\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"sc3\"><span class=\"re1\">&lt;filterchain<span class=\"re2\">&gt;<\/span><\/span><\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"sc3\"><span class=\"re1\">&lt;tokenfilter<span class=\"re2\">&gt;<\/span><\/span><\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"sc3\"><span class=\"re1\">&lt;linetokenizer<span class=\"re2\">&gt;<\/span><\/span><span class=\"re1\">&lt;\/linetokenizer<span class=\"re2\">&gt;<\/span><\/span><\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"sc3\"><span class=\"re1\">&lt;\/tokenfilter<span class=\"re2\">&gt;<\/span><\/span><\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"sc3\"><span class=\"re1\">&lt;sortfilter<span class=\"re2\">&gt;<\/span><\/span><span class=\"re1\">&lt;\/sortfilter<span class=\"re2\">&gt;<\/span><\/span><\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"sc3\"><span class=\"re1\">&lt;uniqfilter<span class=\"re2\">&gt;<\/span><\/span><span class=\"re1\">&lt;\/uniqfilter<span class=\"re2\">&gt;<\/span><\/span><\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"sc3\"><span class=\"re1\">&lt;\/filterchain<span class=\"re2\">&gt;<\/span><\/span><\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class=\"sc3\"><span class=\"re1\">&lt;\/copy<span class=\"re2\">&gt;<\/span><\/span><\/span><br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; <span class=\"sc3\"><span class=\"re1\">&lt;\/sequential<span class=\"re2\">&gt;<\/span><\/span><\/span><br \/>\n&nbsp; &nbsp; <span class=\"sc3\"><span class=\"re1\">&lt;\/macrodef<span class=\"re2\">&gt;<\/span><\/span><\/span><br \/>\n<span class=\"sc3\"><span class=\"re1\">&lt;\/project<span class=\"re2\">&gt;<\/span><\/span><\/span><\/div><\/td><\/tr><\/tbody><\/table><\/div>\n<\/p>\n<h2>Remarks and Caveats<\/h2>\n<ul>\n<li>Contents of all files are sorted by line.<\/li>\n<li>Duplicate lines are removed.<\/li>\n<\/ul>\n<h2>Example<\/h2>\n<p>\nThe example runs on the following directory:<br \/>\n<a href=\"https:\/\/nikostotz.de\/blog\/wp-content\/uploads\/2013\/02\/Bildschirmfoto-2013-02-24-um-19.38.30.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-168\" alt=\"ant-merge-dirstructure\" src=\"https:\/\/nikostotz.de\/blog\/wp-content\/uploads\/2013\/02\/Bildschirmfoto-2013-02-24-um-19.38.30.png\" width=\"222\" height=\"163\" \/><\/a><\/p>\n<p>with the following file contents:<\/p>\n<p><tt>file1.txt<\/tt><\/p>\n<pre>\r\n# random prefix\tfile\tline\tremark\r\n7609\tfile1.txt\t1\r\n24307\tfile1.txt\t2\r\n31889\tfile1.txt\t3\tno newline afterwards\r\n<\/pre>\n<p><tt>file2.txt<\/tt><\/p>\n<pre>\r\n# random prefix\tfile\tline\tremark\r\n5808\tfile2.txt\t1\r\n32706\tfile2.txt\t2\tpreceding empty line\r\n\r\n7848\tfile2.txt\t3\tfollowing empty line\r\n17727\tfile2.txt\t4\r\n3071\tfile2.txt\t5\tone newline afterwards\r\n<\/pre>\n<p><tt>file3.txt<\/tt><\/p>\n<pre>\r\n# random prefix\tfile\tline\tremark\r\n16411\tfile3.txt\t1\r\n10605\tfile3.txt\t2\tfollowed by several newlines\r\n\r\n\r\n\r\n<\/pre>\n<\/p>\n<p>\nOnce we run the Ant script, the resulting file will look as follows:<br \/>\n<tt>merged.txt<\/tt><\/p>\n<pre>\r\n\r\n# random prefix\tfile\tline\tremark\r\n10605\tfile3.txt\t2\tfollowed by several newlines\r\n16411\tfile3.txt\t1\r\n17727\tfile2.txt\t4\r\n24307\tfile1.txt\t2\r\n3071\tfile2.txt\t5\tone newline afterwards\r\n31889\tfile1.txt\t3\tno newline afterwards\r\n32706\tfile2.txt\t2\tpreceding empty line\r\n5808\tfile2.txt\t1\r\n7609\tfile1.txt\t1\r\n7848\tfile2.txt\t3\tfollowing empty line\r\n<\/pre><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Apache Ant has lots of useful built-in tasks. However, a task for merging text files is not among them. Fortunately, we can combine some built-in tasks to achieve this goal. Below, you&#8217;ll find the macro along a simple demo: 123456789101112131415161718192021222324252627282930313233343536&lt;project default=&quot;merge-demo&quot;&gt; &lt;target name=&quot;merge-demo&quot;&gt; &nbsp; &nbsp; &lt;merge tofile=&quot;merged.txt&quot;&gt; &nbsp; &nbsp; &nbsp; &nbsp; &lt;sourcefiles&gt; &nbsp; &nbsp; &nbsp; [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[10,11],"class_list":["post-165","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-ant","tag-merge"],"_links":{"self":[{"href":"https:\/\/www.nikostotz.de\/blog\/wp-json\/wp\/v2\/posts\/165","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.nikostotz.de\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.nikostotz.de\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.nikostotz.de\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.nikostotz.de\/blog\/wp-json\/wp\/v2\/comments?post=165"}],"version-history":[{"count":10,"href":"https:\/\/www.nikostotz.de\/blog\/wp-json\/wp\/v2\/posts\/165\/revisions"}],"predecessor-version":[{"id":176,"href":"https:\/\/www.nikostotz.de\/blog\/wp-json\/wp\/v2\/posts\/165\/revisions\/176"}],"wp:attachment":[{"href":"https:\/\/www.nikostotz.de\/blog\/wp-json\/wp\/v2\/media?parent=165"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.nikostotz.de\/blog\/wp-json\/wp\/v2\/categories?post=165"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.nikostotz.de\/blog\/wp-json\/wp\/v2\/tags?post=165"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}