remove the Carriage return with bash

I have a Ldif file which I would extract some lines from.

the LDIF is 78 character by line, so whenever there is a line contain more than 78 character it's gonna be devided in two line.

what I want is to count the line which contain 78 characters and if so I will remove the carriage return to rebuild my line again.

so I look for a function or any other way to count if the line is 78 characters and remove the carriage return if so.

example:

description: Gestion des acces de la CR 902 sur les environnements Big Data en ecriture
distinguishedName: CN=GU_902_A_D5_IC_W,OU=Groupes,OU=CR 902,OU=CAM,DC=LDSCOMPT ES
instanceType: 4
whenCreated: 20180115131643.0Z

result I need:

description: Gestion des acces de la CR 902 sur les environnements Big Data en ecriture
distinguishedName: CN=GU_902_A_D5_IC_W,OU=Groupes,OU=CR 902,OU=CAM,DC=LDSCOMPTES
instanceType: 4
whenCreated: 20180115131643.0Z

2 Answers

Based on your samples, it sounds like what you actually want to do is append the following line (removing an additional space character) if the current line is exactly 78 characters long:

$ sed '/.\{78\}/{N;s/\n //;}' file.ldif
description: Gestion des acces de la CR 902 sur les environnements Big Data en ecriture
distinguishedName: CN=GU_902_A_D5_IC_W,OU=Groupes,OU=CR 902,OU=CAM,DC=LDSCOMPTES
instanceType: 4
whenCreated: 20180115131643.0Z

A possibly less fragile approach might be to adapt 40. Append a line to the previous if it starts with an equal sign "=" from Peter Krumins' Sed One-Liners Explained to join the following line when it begins with a space

sed -e :a -e '$!N;s/\n //;ta' -e 'P;D' file.ldif

NOTE: if your files use DOS/Windows style CRLF line endings, then you will need to replace \n in the above expressions by \r\n i.e.

sed '/.\{78\}/{N;s/\r\n //;}' file.ldif

or

sed -e :a -e '$!N;s/\r\n //;ta' -e 'P;D' file.ldif

You can use awk for that:

awk '{substr($0,78,78)==""?a="\n":a="";printf "%s"a,$0}'

This command simply checks whether the 78. character of each line is empty and prints the line with or without the line break.

From your example I get that there was also an indention inserted (who does that?!?), to remove these leading extra spaces modify the command to:

awk '{gsub(/^ /,"",$0);substr($0,78,78)==""?a="\n":a="";printf "%s"a,$0}'

If you have unwanted carriage returns in the file remove them as well:

awk '{gsub(/\r/,"",$0);gsub(/^ /,"",$0);substr($0,78,78)==""?a="\n":a="";printf "%s"a,$0}'

Example run

$ <test awk '{gsub(/^ /,"",$0);substr($0,78,78)==""?a="\n":a="";printf "%s"a,$0}'
description: Gestion des acces de la CR 902 sur les environnements Big Data en ecriture
distinguishedName: CN=GU_902_A_D5_IC_W,OU=Groupes,OU=CR 902,OU=CAM,DC=LDSCOMPTES
instanceType: 4
whenCreated: 20180115131643.0Z