Programming Language Generations

The well documented history of programming languages contains some that never existed which were then invented and aptly named as a result. Ada is one of those languages; although Ada Lovelace never saw a machine that Charles Babbage never built; she did write software for it that would in theory calculate Bernouli numbers, and has since been proven to work.(Baum)i Although Ada is a retronym (a term applied after the invention’s creation); there were many languages available and used before ADA, these include ALGOL, APT, FORTRAN and many others. (Piggot)ii

The logical history of software is available from the HOPL, a site devoted to the archival of all programming languages (Piggot)iii; although we could argue that the Jaquard Loom in the late eighteenth century (1890) was the first use of punch cards, and the first programming language to make use of recursion to produce fine rugs; the cards were actually pre-dated by switches when it comes to computers and programming.(Penn)iv

The reasons for abstracting programming language are very simple; machine language is impossible to think in; Humans usually think in the terms of the language they have been taught (New Scientist, 2004)v, not 1’s and 0’s. The amount of errors experienced by early computers during their first runs was incredible (cnet, 2008)vi; however a good portion of these errors were the result of a human whom does not understand how to program the machines memory, or mechanical issues; this paradigm has not changed much since the first days of computing, as we humans have not evolved much in the last 50 years; the languages used to program the raw data and computer hardware have.

Mechanical gears and vacuum tubes have given way to semiconductors; transistors, capacitors and resistors that are billionths of a human hair in size and consume a modicum of electricity whilst functioning. This almost three orders of magnitude increase in compute power has brought about its own issues; since there are no moving parts in a modern computer the potential for mechanical failure has been replaced by the potential for the failure in software: where the complexity arising from moving parts may be equated to lines of code.(Lew et al.)vii

The term debugging was invented at the U.S. Naval research centre at Harvard university when Rear Admiral Grace Murray Hooper noted issues with a program running on a Mark II computer; and then removed a moth from one of the relay’s coining the term debugging in the process.(Time, 1984)viii

The modern computer has a gargantuan amount of storage for both core memory and non-volatile memory when compared with the computers Grace Hooper used. To imagine how to program in machine language we would first have to consider our underlying systems architecture; this is as defined by the vendor of the system in question and is referred to as the Machine Language. (IBM, 1998)ix The most commonly sold personal computer today uses a machine language or instruction set architecture; referred to as x86 and x86-64; x86 has been around a long time; however as a language the 32 bit version has over 300 separate instructions for integer, floating point and other operations, combined with 32-bit width and register addresses and the terabytes for storage that 32-bits can address; the task quickly becomes impossible to program the system manually( Intel)x; although some circles still do (Wikipedia)xi; So programming languages arose as the result of machine language complexity.

Enter the “compiler”; software program to take and abstracted human readable programming language and turn it into machine useable code. The first compiler ever used was the NELIAC (HOPL)xii and it compiled ALGOL. It is assumed that the early compilers for Fortran, PASCAL and other languages were hand made that is designed to transform instructions by the hands of some very talented programmers; modern compilers are compiled by other compilers; That is a newer version of a compiler will utilize an older version of the same compiler to compile itself and once compiled may compile other software as needed; Examples include the GNU’s GCC compiler (GNU)xiii, or any commercially available compiler.

Programming paradigms have evolved in conjunction with the underlying technology accordingly; the current “Object-Oriented” paradigm evolved because we had developed computers that had enough resources to devote to a compiler to reduce the amount of time required to create software to run programs with our computers but sacrificing program effiencey. The previous common paradigm was a functional one for high level languages; such as ALGOL, FORTRAN or PL/1; the move to imperative programming languages was a logical transition from the functional paradigm since thinking in mathematical abstraction is more difficult than natural language; imperative languages like BASIC or SmallTalk were the result. The development of the Obejct-oriented paradigm is a result of using imperative object based programming languages, such as C and then once discovering their limitations in large programming environments developing C++ and the Object oriented standard to reduce programming time.

The measure of a computers overall efficiency is the time it takes the computer to do a given task; this is at the heart of benchmarking computer systems (University of Tennessee)xiv; however the time required to produce code is measured in man hours; (Anselmo et al.)xv, since the primary cost of developing software is the human developing it; the reduction of time spent by the programmer to create a program is money saved by the company making said software; and thus forms the basis of the Programming Language market, which is a large business.

Rapid Application Development and Extreme programming are methodologies that allow the development of application features and the reliability of said application to be of paramount importance in both its development and support; the current paradigm is shifting in favour of a methodology driven environments where project management methods determine controls for programming projects, and the underlying languages become less important. Current trends in programming include the move to interpretive, object-oriented, interactive & distributed programming languages such as Python, PHP, C# and .NET, although C/C++ and JAVA will be around for decades to come more a result of software market pressure than evolution: Where existing requirements stipulate that development time is minimal and the programs created serve a modular function such as Content Management Systems with native database functionality; we are also seeing trends in object oriented database access in addition to traditional variable storage methods; were applications are created with a database attached to them to improve performance and functionality. There is also the move online to standards such as AJAX(Opendirectory)xvi/SOAP (w3c)xvii where formal data interchange is supported by ubiquitous standards between unrelated third party organizations; the best example of this is Facebook, it’s programmed in PHP and uses AJAX and SOAP to allow third party companies to access people’s vital data. The social ramifications of this endeavor are still being rectified.

We have yet to produce a computer that can code it’s own programs; although LISP has many applications that can re-code their own functions and variables according goals and neural models(Luger)xviii we have yet to see an application that may produce programs itself as needed by objective alone; although many experiments have been completed, automating programming is one of the greatest challenges to the world of artificial intelligence, we can even purchase software that will aid in the modeling and selection & classification of unknowns but we cannot find applications that code themselves. (Ward Systems)xix

---------------------------------------------------------------------------------------------------------------
i Baum, Joan (Archon Books, December 1986). The Calculating Passion of Ada Byron.
ii Pigot, Diramud (Murdoch University, 2006) HOPL: an interactive Roster of Programming Languages, [Online] World Wide Web, Available From:
http://hopl.murdoch.edu.au/
(Accessed on October 20th 2008)
iii Pigot, Diramud (Murdoch University, 2006) HOPL: an interactive Roster of Programming Languages, [Online] World Wide Web, Available From:
http://hopl.murdoch.edu.au/
(Accessed on October 20th 2008)
iv Penn University (n.d.) Online Eniac Museam [Online] World Wide Web, Available From:
http://www.seas.upenn.edu/~museum/3tech.html
(Accessed on October 20th 2008)
v Biever, Celeste (August 19th 2004) Language May Shape Human Thought [Online] World Wide Web, Available From:
http://www.newscientist.com/article.ns?id=dn6303
(Accessed on October 20th 2008)
vi Kanellos, Michael (Cnet, February 13th 2006) Eniac a computer is Born [Online] World Wide Web,
Available From:
http://news.cnet.com/ENIAC-A-computer-is-born/2009-1006_3-6037980.html
(Accessed on October 20th 2008)
vii Lew, K.S. ; Dillon, T.S.; Foreward, K.E.(IEEE Transactions on Software Engineering, November 1988 Vol 14, no. 11 P1645 – 1655) Software Complexity and it’s Impact on Software Reliability, [Online] World Wide Web, Available From:
http://www2.computer.org/portal/web/csdl/doi/10.1109/32.9052
(Accessed on October 20th 2008)
viii Taylor, Alexander (Time April 16th 1984) The wizard inside the machine [Online] World Wide Web,
Available From:
http://www.time.com/time/printout/0,8816,954266,00.html
(Accessed on October 20th 2008)
ix Super Computer Education & Resarch Centre Indian Institute of Science (IBM, n.d.) Online Glossary of Terms [Online] World Wide Web,
Available From:
http://www.serc.iisc.ernet.in/ComputingFacilities/systems/cluster/vac-7.0/html/glossary/czgm.htm
(Accessed on October 20th 2008)
xSmith, Zack (Smith, 2005) The Intel 8086/8088/80186/80286/80386/80486 Instruction Set. [Online] World Wide Web, Available from:
http://home.comcast.net/~fbui/intel.html
(Accessed on October 20th 2008)
xi Wikipedia (n.d) Demo (Computer Programming), [Online] World Wide Web, Available From:
http://en.wikipedia.org/wiki/Intros#Intros
(Accessed on October 20th 2008)
xii Pigot, Diramud (Murdoch University, 2006) Navy Electronics Laboratory International ALGOL Complier [Online] World Wide Web Available From:
http://hopl.murdoch.edu.au/showlanguage2.prx?exp=32
(Accessed on October 20th 2008)
xiii Free Software Foundation (July 31st 2008) The GNU Compiler Collection (GCC) [Online] World Wide Web, Available From:
http://gcc.gnu.org/
(Accessed On October 20th 2008)
xiv Petitet, A; Whaley, R. C.; Dongarra, J. ; Cleary, A. (University of Tennesee, September 10 2008) HPL - A Portable Implementation of the High-Performance Linpack Benchmark for Distributed-Memory Computers, [Online] World Wide Web, Available From:
http://www.netlib.org/benchmark/hpl/
(Accessed On October 20th 2008)
xv Anselmo, Donald; Ledgard, Henry (ACM Proceedings, November 2003) Measuring Productivity in the Software Industry, [Online] PDF Document, Available From:
http://portal.acm.org/citation.cfm?doid=948383.948391
(Accessed on October 20th 2008)
xvi OpenDirectory (Netscape, 2007) Programming Lanaguages – AJAX ,[Online] World Wide Web
Available From:
http://www.dmoz.org/Computers/Programming/Languages/JavaScript/AJAX/
(Accessed on October 20th 2008)
xvii Box, Don; Ehenbuske, David; Kakivaya, Gopal; Layman, Andrew; Mendelshon, Noah; Nielson, Frystyk, Henrik; Thatte, Satish; Winer, Dave (W3C Note, May 2000) Simple Object Access Protocol (SOAP) 1.1, [Online] World Wide Web, Available From:
http://www.w3.org/TR/2000/NOTE-SOAP-20000508/
(Accessed on October 20th 2008)
xviii Phd. Luger, George (Pearson, 2009) Artificial Intelligence: Structures and Strategies for Complex Problem Solving, 6th Edition, [Online] PDF Document, Available From:
http://www.cs.unm.edu/~luger/ai-final/supplement-book-preface.pdf
(Accessed On October 20th 2008)
xix Ward Systems (Wardsystemsgroup, 2007) Advanced Neural Network and Genetic Algorithm Software Information Page [Online] World Wide Web Available From:
http://www.wardsystems.com/
(Accessed on October 20th 2008)

How to increase storage subsystem speed (without increasing disk rotation)

The primary reason that physical disks are around two orders of magnitude slower than the registers and cache of a central processing unit is the simple fact that one relies on physical motion, whilst the other utilizes waves of electrons operating at speeds close to the speed of light. The only limiting factor for the speed of a wave of electrons traveling through a semi-conductor is the material and its fabrication size this is often referred to as the fabrication “Process” used. In turn the process produces the physical limitations of the solid state circuitry.

The theoretical mathematical limitation is described by Taylor and Wheeler (1992)i to be 2L/c where L is the average distance to the memory and c is the maximum celerity.

The primary limiting factor within a central processing unit is propagation delay as well as the distance to the physical memory. Since the next limitation is the core oscillator that acts as the CPU’s clock we see that there are no moving components and the physical limitations of the medium being used impose a speed limit to the clock itself. When room temperature superconductors become a reality, then the only limitation to compute speed will be the mediums permittivity, and information will travel at the speed of light (C ).

When a computing machine accesses physical storage a section of the memory mapped to the devices controller is used as the interface; that interface in turn has to utilize the Hardware Abstraction Layer and Operating System (including all of it’s dependencies) to locate the appropriate driver which then utilizes the native commands to send and receive information from core memory to non-volatile storage via the storage device controller. One order of magnitude of latency is introduced when the core memory is utilized; the second order of magnitude of latency is between the controller and the storage device itself.(Developer Shed, 2004)ii The average core memory latency is measured in microseconds the average hard drive latency is measured in the milliseconds. (Storage Review, 2005)iii

Current methods utilized to increase storage seek, read & write performance are being implemented by utilizing serial buses versus parallel ones(Dorian Cougias, 2003)iv, employing techniques such as command queuing (Patrick Schmidt, 2004)v to offer a type of “branch prediction” to the drive itself and increasing local storage buffer size on the drive so less time is spent physically looking for bits, as well as utilizing multiple drives in a RAID configuration to aggregate available I/O bound bandwidth by utilizing specialized hardware or software, this also includes “Storage Area Networks” which in essence are local drives put somewhere else with gargantuan amounts of input & output bandwidth between the drives and computers that use them.

Performance increases to operational systems latency involves two kinds of improvements, the first family require no hardware modification and fall into the category of software optimizations, the second kind require hardware configuration and architectural changes.

Here’s a list of known Optimizations that will improve storage latency within the Intel PC architecture that I have gained from my personal experience:

1. Utilize a file system layout that puts the most frequently accessed files closest to the spindle

2. Utilize an optimized chunk size within said file system and its application, this is a delicate endeavor, some debate its validity, vendors such as Google use GFS that has file chunks that are 64MB in size; conversely the default NTFS chunk is 64KB; GFS is optimized for web crawling & reliability on commodity hardware. (Ghemawat et al, 2003)vi

3. Ensure that the drive has a dedicated bus; for parallel ATA systems this meant purchasing extra controller hardware, it’s a standard in serial ATA and serial SCSI storage controllers.

4. Optimize all controller driver software to the most current stable version available including the drives and the controller’s firmware.

5. Within the software driver offload as many storage calculations to dedicated systems hardware; these are usually part of the driver options or bios settings and may be implemented within either the system or the controller itself.

6. If multiple disks are available configure a RAID array; again depending on application, two drives connected in a raid 0 array may easily achieve twice the write and read performance; with half the reliability!

7. If a page file is used ensure it’s a static otherwise unless required remove the page file.

Now for architectural changes that would improve storage latency; to increase overall system performance and reduce the amount of primary latency involved the speed of the front side bus or the core memory bandwidth is the first place all systems vendors work to improve. Increasing the available core memory bus bandwidth and I/O latency improves storage latency as it’s the first caveat within systems architecture. Architectural changes are relatively expensive when compared with optimizations and usually take time to implement as the storage vendor consortium must adopt them as manufacturing standards; thus unless they are developed by that consortium or these optimizations are more cost effective than a current technology they will not usually get implemented in the public domain.

The second place to increase available Input output (I/O) bandwidth is the system drive’s bus itself. The current speed of Serial ATA is 300MBs/3Gbps this is achieved across a four pin serial interface, the next generation of serial ATA will be capable of 600MB/s or 6.0Gbps(Leroy Davis, 2008)vii, the increased bus speed will require that future drives have larger local buffer memories and better command queuing. So the next methods to increase drive performance without modifying disk’s rotational speed are as follows:

1. Increase aerial storage density and reduce platter diameter, utilize new magnetic substrates with higher potential well resolutions and smaller drive heads.

2. Increase the drive’s communications buffer size, preferably by orders of 2 (128MB, 256MB, 512MB…), thus reducing the amount of seek, read and write commands actually issued to the platter.

3. Increase the drive controller’s input/output bandwidth on both sides of the controller, eg: from the drive to the controller and from the controller to core memory via the driver and operating system; including increasing the controller’s bus clock rate.

Although this discussion question refers to physical disk storage, there is a trend emerging for non-volatile solid state storage based upon NAND flash technology; IBM also has an organic substrate that has shown promise with areal densities far higher than physical disks entitled the millipede.(Vettiger et al, 1999)viii although advanced concepts such as holographic storage, AFM storage and others have been around for a long time they are yet to be cost effective enough to be adopted as non-volatile storage solutions by industry.

i Edwin F. Taylor, John Archibald Wheeler (1992) Spacetime Physics, 2nd ed. United States: W.H. Freeman & Co

ii Jkbaseball, Developer Shed (2004-11-30) Effects of Memory Latency [Online] World Wide Web, Available From:

http://www.devhardware.com/c/a/Memory/Effects-of-Memory-Latency/

(Accessed on Oct 1st 2008)

iii Charles M Kozierok, Storage Reivew (2005) The PC Guide – Latency [Online] World Wide Web, Available From:

http://www.storagereview.com/guide2000/ref/hdd/perf/perf/spec/posLatency.html

(Accessed on Oct 1st 2008)

iv Dorian Cougias, Search Storage (2003) The advantages of Serial ATA over Parallel ATA [Online] World Wide Web, Available From:

http://searchstorage.techtarget.com/tip/1,289483,sid5_gci934874,00.html

(Accessed on Oct 1st 2008)

v Patrick Schmid, Toms Hardware (Nov 16 2004) Can Command Queuing Turbo Charge SATA Hard Drives? [Online] World Wide Web, Available From:

http://www.tomshardware.com/reviews/command-queuing-turbo-charge-sata,922.html

(Accessed on Oct 1st 2008)

vi Sanjay Ghemawat, Howard Gobioff, Shun-Tak Leung, 19 ACM Symposium on Operating Principles (Lake George New York, 2003) The Google File System [Online] World Wide Web, Available From:

http://labs.google.com/papers/gfs.html

(Accessed on Oct 2nd 2008)

vii Leroy Davis, Interface Bus (Sep 17 2008) Serial ATA Bus Description [Online] World Wide Web, Available From:

http://www.interfacebus.com/Design_Connector_Serial_ATA.html

(Accessed on Oct 1st 2008)

viii Vettiger et al, IBM Journal of Resarch and Development (1999),The millipede more than one thousand tips for the future of AFM data storage [Online] World Wide Web, Available from:

http://www.research.ibm.com/journal/rd/443/vettiger.html

(Accessed on Oct 2nd 2008)