It seems like with advent of technology, the computing world grows closer and closer to nature everyday. The biggest trend which exhibits this facts is the continual need for a developer to learn new technologies and new languages.
It was only a few decades ago where a single degree or a diploma could get you through a lifetime. Just ask your parents. Most parents, like mine would probably have a bachelor's degree which allowed them to provide themselves and for their kids. This includes not just the needs for food, basic education, and 'wants', but also in my case, higher education. I was one of those fortunate enough to have parents who supported my endeavor to pursue a master's degree. But to top it all, they managed to keep their heads above water without falling into the endless abyss of loans.
Today, I hear countless stories and have encountered people hesitant to even think about higher education due to the cost, and struggle between quality and quantity.
Even with my high school, bachelor's, and master's degree, I still have to learn new tricks. It's not because I need to. Need is immediate. Not learning is slow extinction.
And to prevent extinction, I decided to learn Python. Some call it the language of and for hackers. I don't portray myself as a hacker. I like perfection way too much. Which prevents me from slapping things together to call it a day and never look at it again. For python, I started a few weeks ago by reading some basic tutorials. Reading those tutorials was pretty boring, but life was interesting again when I was presented with a challenge at work.
The challenge presented itself as a need to insert a few additional characters in an already existing file. The file format is a Motorola S-Record (.mot file) and is fully human readable. It's contents include a bunch of numbers which are machine instructions, codes that your computer/microcomputer can understand. Without getting into too much detail about the S-Record, here's how one would look like:
S315FFFF00007FB8FBEA0004FD68E2FBEA0005FD68EA4C
And here's what the challenge intended to make it look like.
{ 0x15, 0xFF,0xFF,0x00,0x00, 0x7F,0xB8,0xFB,0xEA,0x00,0x04,0xFD,0x68,0xE2,0xFB,0xEA,0x00,0x05,0xFD,0x68,0xEA, 0x4C, },
To someone who is a computer literate, there are three basic ways of doing it,
1. Manually using text editors
2. Semi-automatic methods i.e. Use text import wizards to import and partially format the data and then fill out the unfilled blanks.
3. Fully automatic methods. Write a computer program which does everything.
Well, #1 and #2 would work if there were a limited number of these S-Records, but usually a .mot file contains at least several hundred of these. The hazards of human intervention are high, the task is simple, which makes #3 highly attractive. And the fact that the challenge doesn't have any constraints on computing power or speed makes it a perfect candidate for creating a python script with a purpose. And here's the not-so-sexy, and not-so-fast script that does exactly what is needed.
It took me about 8 hours to get it work, even after borrowing some initial code, but it works and helps me from not becoming extinct ;) So Python has definitely made it's way onto the horizon. Here's to hope that there's more to it in the future.
It was only a few decades ago where a single degree or a diploma could get you through a lifetime. Just ask your parents. Most parents, like mine would probably have a bachelor's degree which allowed them to provide themselves and for their kids. This includes not just the needs for food, basic education, and 'wants', but also in my case, higher education. I was one of those fortunate enough to have parents who supported my endeavor to pursue a master's degree. But to top it all, they managed to keep their heads above water without falling into the endless abyss of loans.
Today, I hear countless stories and have encountered people hesitant to even think about higher education due to the cost, and struggle between quality and quantity.
Even with my high school, bachelor's, and master's degree, I still have to learn new tricks. It's not because I need to. Need is immediate. Not learning is slow extinction.
And to prevent extinction, I decided to learn Python. Some call it the language of and for hackers. I don't portray myself as a hacker. I like perfection way too much. Which prevents me from slapping things together to call it a day and never look at it again. For python, I started a few weeks ago by reading some basic tutorials. Reading those tutorials was pretty boring, but life was interesting again when I was presented with a challenge at work.
The challenge presented itself as a need to insert a few additional characters in an already existing file. The file format is a Motorola S-Record (.mot file) and is fully human readable. It's contents include a bunch of numbers which are machine instructions, codes that your computer/microcomputer can understand. Without getting into too much detail about the S-Record, here's how one would look like:
S315FFFF00007FB8FBEA0004FD68E2FBEA0005FD68EA4C
And here's what the challenge intended to make it look like.
{ 0x15, 0xFF,0xFF,0x00,0x00, 0x7F,0xB8,0xFB,0xEA,0x00,0x04,0xFD,0x68,0xE2,0xFB,0xEA,0x00,0x05,0xFD,0x68,0xEA, 0x4C, },
To someone who is a computer literate, there are three basic ways of doing it,
1. Manually using text editors
2. Semi-automatic methods i.e. Use text import wizards to import and partially format the data and then fill out the unfilled blanks.
3. Fully automatic methods. Write a computer program which does everything.
Well, #1 and #2 would work if there were a limited number of these S-Records, but usually a .mot file contains at least several hundred of these. The hazards of human intervention are high, the task is simple, which makes #3 highly attractive. And the fact that the challenge doesn't have any constraints on computing power or speed makes it a perfect candidate for creating a python script with a purpose. And here's the not-so-sexy, and not-so-fast script that does exactly what is needed.
#!/usr/bin/python
## Creates data records as a c-struct from lines in a MOT-file. The infile has to be *.MOT.
## Script is now intelligent and splits records which cross 1k boundaries into two records and
## inserts a dummy checksum into the two split records
## Created: Onkar Raut, onkar.raut@renesas.com, 2014, Oct, 06th.
## Usage: 'create_data_record.py -i <path_to_input_file> -o <path_to_output_file>'
## OR 'create_data_record.py -h'
## Example: python create_data_record.py -i TestImage.mot -o TestImage.dat
## Argument parsing code is borrowed from: http://www.tutorialspoint.com/python/python_command_line_arguments.htm
import sys, getopt
def main(argv):
inputfile = ''
outputfile = ''
try:
opts, args = getopt.getopt(argv,"hi:o:",["ifile=","ofile="])
except getopt.GetoptError:
print 'create_data_record.py -i <path_to_input_file> -o <path_to_output_file>'
sys.exit(2)
for opt, arg in opts:
if opt == '-h':
print 'create_data_record.py -i <path_to_input_file> -o <path_to_output_file>'
sys.exit()
elif opt in ("-i", "--ifile"):
inputfile = arg
elif opt in ("-o", "--ofile"):
outputfile = arg
print 'Input file is "', inputfile
print 'Output file is "', outputfile
f_wr = open(outputfile, 'w+')
with open(inputfile, 'r') as f_rd:
for line in f_rd:
nr_bytes_remain = 0
##Strip out the first two bytes in the line##
line = line[2:]
##Create size information##
nr_bytes = int(line[:2],16)
nr_bytes_orig = nr_bytes+1
size = 2*nr_bytes
size_orig = size
##Pull in the address being used##
address = int(line[2:10],16)
nr_data_bytes = nr_bytes - 4 - 1
nr_data_bytes_orig = nr_data_bytes
## Check if address exceeds a 1k boundary
if ((address + nr_data_bytes) > ((address&0xFFFFFC00)+1024)):
##Print some debug info
print '******Detected need to insert additional data record ******'
##Calculate the number of bytes to put in this data record
print 'Block boundary = ',hex((address&0xFFFFFC00)+1024)
print 'Address = ',hex(address)
nr_data_bytes = ((address&0xFFFFFC00)+1024) - address
nr_bytes_remain = nr_data_bytes_orig - nr_data_bytes
nr_bytes = nr_data_bytes + 4 + 1
size = 2*nr_bytes
print 'Num of additional bytes in record = ',nr_bytes_remain
print '\n'
##Print the size of the read record
print 'Address = ',hex(address)
nr_data_bytes = nr_bytes - 4 - 1
print 'Num data bytes in record = ',nr_data_bytes
##Format the data record##
i = 0
newline = '{ ';
if nr_bytes_remain > 0:
##Fill in the new number of bytes filled in##
newline = newline + str(hex(nr_bytes)) + ', /*ADJ*/ '
size -= 2
i = 2
## Fill in the record ##
while i<=size:
newline = newline + '0x' + line[0+i:0+i+2] + ','
if i == 0:
newline = newline + ' '
elif i == 8:
newline = newline + ' '
elif i == (size - 2) and nr_bytes_remain == 0:
newline = newline + ' '
i+=2
if nr_bytes_remain > 0:
newline += ' /* Dummy */ 0xAA, '
newline += ' },\n'
f_wr.write(newline)
if nr_bytes_remain > 0:
print 'Adjusted data record at current address as shown below '
print newline
if nr_bytes_remain > 0:
addline = '{ '
##Fill in the number of bytes left over from prev record + address + checksum
addline = addline + str(hex(nr_bytes_remain + 4 + 1)) + ', /*NEW*/ '
block_boundary = ((address&~0x3FF)+1024)
## Address on a block boundary ##
block_boundary_b1 = ((block_boundary >> 24) & 0xFF);
block_boundary_b2 = ((block_boundary >> 16) & 0xFF);
block_boundary_b3 = ((block_boundary >> 8) & 0xFF);
block_boundary_b4 = ((block_boundary >> 0) & 0xFF);
addline += str(hex(block_boundary_b1)) + ',' + str(hex(block_boundary_b2)) + ',' + str(hex(block_boundary_b3))+ ',' + str(hex(block_boundary_b4)) + ', '
## Fill in the record ##
while i<size_orig:
addline = addline + '0x' + line[0+i:0+i+2] + ','
if i == 0:
addline = addline + ' '
elif i == 8:
addline = addline + ' '
elif i == (size - 4):
addline = addline + ' '
i+=2
addline += ' /*Dummy*/ 0xAA },\n'
print 'New record created below with remaining bytes = ', nr_bytes_remain, ' at address ', hex(block_boundary)
print addline
f_wr.write(addline)
f_rd.closed
f_wr.close();
if __name__ == "__main__":
main(sys.argv[1:])
