Oral-History:Herbert Freeman

About Herbert Freeman

Herbert Freeman

Herbert Freeman was born in Frankfurt-am-Main, Germany on 13 December 1925. His family moved to the United States in 1936, but Freeman entered the country only in 1938 after going through several visa rejections. Freeman got his Bachelors in Electrical Engineering at Union College and started his career at the Sperry Gyroscope Company, refining analog computers of the period and designing the company’s first digital computer. He earned his Doctorate degree in 1956 from Columbia University, while working at Sperry. Leaving his twelve-year career at Sperry behind, Freeman began teaching at New York University and became head of the electrical engineering department in ’68. While he was at NYU, Freeman gradually shifted his interest toward computer graphics. He also changed the electrical engineering department name to the “electrical engineering and computer science” department. As a result of financial difficulties the university went through, the engineering school was merged with the Polytechnic Institute of Brooklyn. A small group of engineering faculty decided not to follow the move but instead form an applied science division at NYU. However, disappointed by the political struggle and uncertainties, Freeman eventually left NYU for Rensselaer Polytechnic Institute in 1975.

Herbert Freeman took strong interest in computer graphics. He published numerous works on geometric patterns from ’61 and was a co-founder with Azriel Rosenfeld of the University of Maryland of the Journal of Computer Graphics and Image Processing in ’70. Later it changed its name to Computer Graphics, Vision, and Image Processing Journal, and with the expansion of the industry, the journal was split into two in the 1990s. In the latter part of the interview, Freeman explains a clear distinction between computer science and computer engineering and states his involvement in map data processing. In the early 1980s he succeeded in developing an automatic labeling system for map data, a highly complex process for computers. The interview concludes with Freeman’s work as president of the International Association for Pattern Recognition, his position as director of the Center for Computer Aids for Industrial Productivity at Rutgers, and finally as president of MapText, Inc., a firm he founded to commercialize software for the automatic labelng of maps.

Freeman was named a 1967 IEEE Fellow "for contributions to the theory of multivariable discrete-time systems and digital computer graphics."

About the Interview

HERBERT FREEMAN: An Interview Conducted by Frederik Nebeker, Center for the History of Electrical Engineering, 25 June 1998

Interview #339 for the Center for the History of Electrical Engineering, The Institute of Electrical and Electronics Engineers, Inc.

Copyright Statement

This manuscript is being made available for research purposes only. All literary rights in the manuscript, including the right to publish, are reserved to the IEEE History Center. No part of the manuscript may be quoted for publication without the written permission of the Director of IEEE History Center.

Request for permission to quote for publication should be addressed to the IEEE History Center Oral History Program, 39 Union Street, New Brunswick, NJ 08901-8538 USA. It should include identification of the specific passages to be quoted, anticipated use of the passages, and identification of the user.

It is recommended that this oral history be cited as follows:

Herbert Freeman, an oral history conducted in 1998 by Frederik (Rik) Nebeker, IEEE History Center, New Brunswick, NJ, USA.

Interview

Interview: Herbert Freeman

Interviewer: Frederik Nebeker

Date: 25 June 1998

Place: Piscataway, NJ

Childhood, family, and education

Nebeker:

You were born in Frankfurt, 13th December 1925?

Freeman:

Yes, in Frankfurt-am-Main, Germany.

Nebeker:

Can you tell me a little about your family?

Freeman:

My father was a physician in Frankfurt. We have a family tree that shows that we have lived in that part of Germany for many hundreds of years. Then when the Nazi period started in 1932-33, it became uncomfortable for us and other members of my family as we are Jewish, and we started to look toward leaving Germany. Then by 1935 my dad contacted a brother who had come to the United States many years earlier, in 1908 I believe, and was living here. He asked my uncle to sponsor us because you needed a sponsor to get into the United States.

Nebeker:

Were you always interested in radio and technical matters?

Freeman:

I was always interested in technical matters, and I distinctly remember in kindergarten (at age of 5), where I was always tinkering with things, somebody asking me, “What do you want to be when you grow up?” I answered, “an electrical engineer.” I have never veered from that.

Nebeker:

Were other family members engineers?

Freeman:

No. But there was a family friend who was an electrical engineer who impressed me. I believe he was in the telephone business. Also the toys I liked always were technical.

Nebeker:

Did you build a crystal radio?

Freeman:

No, but lots of other gadgets that I took apart and put together again (and some that I was unable to put together again.) I was the typical kind of technically oriented youngster. So yes, that always was my interest. My dad was not very technically oriented, but I had uncles who were. My grandfather was a watchmaker and very handy in tinkering with things.

Nebeker:

Did you know him?

Freeman:

Yes, very well. So there definitely was a connection of some kind.

Immigration to the U.S., 1938

Nebeker:

You suggested it wasn’t a simple story of going from Frankfurt to the United States in ‘38.

Freeman:

It’s a very complicated story. In fact just about a week ago somebody else came and interviewed me about that. I don’t know how much you want to me to get into it.

Nebeker:

Maybe a one-paragraph summary, if that’s possible.

Freeman:

Yes. I’ll tell you, and then you can decide whether you want to leave that out. There is actually one very interesting aspect to it. The problem was, of course, getting a visa to the United States, and in those days it was the Depression. Visas were difficult to get. Immigrants at that point, in the ‘30s, were rather discouraged. My dad applied for a visa, received it, and then came first to get started. He had to take an English proficiency exam in the United States so that he could get established and practice medicine here. He came first, and then the idea was that the family would follow about six months later.

Nebeker:

What year was it that your father came?

Freeman:

<flashmp3>339 - freeman - clip1.mp3</flashmp3>

My dad left Germany in December 1935. I had just turned 10 at that point. Then the family was going to follow about six months later. Then in the spring of 1936 my mother and the two children went to the American Consulate to apply for the visa. It was granted to her and my brother, but not to me. The reason being, supposedly, that I was suffering from tuberculosis. This was a complete surprise to my parents who were a middle-class family and lived in a good neighborhood. It didn’t seem likely.

Immediately my mother had me examined by colleagues of my dad, and they said, “No, there is no evidence of tuberculosis. It must be a mistake.” It was decided that it had to be a mistake, and we were told to apply again in three months. But my mother was anxious to leave Germany, so it was decided I would stay with my grandparents. I lived with that grandfather I mentioned earlier. My mother and brother went ahead, and since it was obviously a mistake, the plan was to get my visa three months later. So they left for the US. Then three months later an aunt of mine, I believe, took me to the Consulate to apply again, and I was again turned down.

At that point my parents were getting more concerned. They got all kinds of evidence together about my other examinations. I was even placed in a hospital for three days for very rigid examination and every conceivable test and x-rays. Some of the results were sent to the United States, where my dad took them to both a hospital in New York City and to the Public Health System in Washington to show that there was absolutely no evidence of tuberculosis. I went two more times to the Consulate, each time at 3-month intervals, and was turned down each time. The conclusion was that the only way to ever get out was to apply in some other Consulate. But that wasn’t so simple because other Consulates would try to steer you to the one you came from. Let’s say you went to France or Switzerland and applied to the American Consulate there, the Consulate would say, “Well, but you are from Germany. You have to apply through Germany. I cannot process you here.” My parents then requested special permission for that, which took some time and eventually was granted, because I think the people in Washington smelled that there was something not quite right going on.

In the meantime this had become a fairly big case. A lot of famous people had weighed in, including Senator Robert Wagner, of the Wagner Labor Act, some famous judges, and others. They wrote letters to Washington and so on. Ultimately my dad got permission for me to apply from Switzerland. An aunt brought me to Switzerland, where I boarded with a family. I then applied in Switzerland and I got my visa and came to the United States in March 1938. Since you asked that much, I will show you some interesting documentation. About ten years ago or so my father-in-law said to me, “You know, they have all these records in Washington. Maybe if you write there you can get copies of some of the letters.” I didn’t know how to go about it, so I wrote to Senator Lautenberg and asked for his help. I have a copy here of the letter that I wrote to him, and then he wrote back very nicely, “I’m sure we’ll get what you’re looking for,” and eventually I got a letter from the National Archives.

Nebeker:

Would it be possible for us to get copies of these?

Freeman:

Yes. The response that came a few months later said, “Yes, we have the letters.” Here is one of them.

Nebeker:

Is this a letter from Albert Einstein about this case?

Freeman:

Yes, and I even have the handwritten draft. It wasn’t that he just gave it to a secretary and said “Write a letter.” He actually drafted it himself, and I have the draft of it. I thought, “It looks like he wrote this, not somebody else.” I sent it to a place in Boston where they are collecting Einstein’s papers. I said, “I have a letter here. There is reason to believe it might be in Einstein’s handwriting. Would you let me know?” They wrote back immediately and said there is absolutely no question that it is in his handwriting.

Nebeker:

It is really remarkable that even Einstein got involved in your visa fight.

Freeman:

Well, it had become a kind of cause celebre. Here’s another letter. This one is in German. Einstein actually wrote three letters, and it wasn’t until the third letter that the ice broke.

Nebeker:

Fine Hall is now Jones Hall at Princeton. I told you I got my degree there, and it’s right down the hall from The Institute for Advanced Study on the Princeton campus.

Nebeker:

Did you ever learn a better explanation of why it was so difficult to get the visa?

Freeman:

It depends on whom you talk to. There are two possibilities. One, that it was an honest mistake at first and that the doctors at the consulate were just unwilling to admit they had made a mistake, and two, that somebody was looking for a payoff, and thought, “Here is a doctor, middle-class family, they must have money. Let’s hold back the little one.” That’s most likely what occurred. But you know this is a curious thing, the idea of payoffs was something that was just totally unknown in Central Europe in the 1930s.

My dad said he later realized that there were many instances in the US where you had to do this to get something done. You had to slip someone money whether it was a waiter at a restaurant or someone else. For example, my father had to take his driver’s test and he had taken some lessons with a driving instructor, and the instructor said, “Okay, we’ll leave this book here and when you take your test put a five-dollar bill in it.” When after the test the instructor came back and opened the book, the money was gone. The instructor said “that means you passed.” It may be better today, but in the ‘30s that was not so unusual apparently.

Nebeker:

When did you finally come to the United States?

Freeman:

March 4, 1938.

Education in the U.S.

Nebeker:

Did you speak English at that time?

Freeman:

Very, very little. I had learned some in school, and, of course, I had tried to learn a little extra knowing that I was coming to the United States. But because it was school learning I basically could not speak it. By the way, I have an interesting comment on all this discussion about bilingual education going on these days. I was put in a regular class, and I remember my parents even thought that I should go to a class of younger children so that it would be easier for me, and then in time move up. The principal specifically said, “No, I think he should go into his age group.”

The first six months I had no idea what was going on. When the other kids got up, I got up; when they sat down, I sat down. Sometimes the teacher would ask a question of me, and I just sat quietly. After six months it started to come through to me, and then after a year I would say I was pretty much in command. Then things moved forward very well. Two-and-a-half years later there was an all-school English essay contest and I won first prize - in English!

Nebeker:

Was this a New York City school?

Freeman:

No, we had moved to Upstate New York, to Waterford, New York. It’s a village north of Albany. A few years later we moved to Cohoes, NY, a city of some 20,000, also north of Albany. I graduated from the Cohoes High School, and then went to Union College in Schenectady, NY.

Nebeker:

Did you want to be an electrical engineer since you were in kindergarten?

Freeman:

Yes, and it was also known in my family. They were always saying, “he’s going to be an electrical engineer.”

Nebeker:

Did you do well at Union College?

Freeman:

Yes, I graduated at the top of my class. I was also Valedictorian at my high school.

Graduate studies; Sperry employment

Nebeker:

What did you do after receiving your Bachelors in Engineering in ‘46?

Freeman:

I came to Columbia University for graduate school and got a Master’s degree. I was a teaching assistant for two years; so it took two years instead of one year to get the degree. Then when I graduated in June ‘48 I went to work at the Sperry Gyroscope Company, in Great Neck, L.I., NY.

Nebeker:

What type of engineering did you specialize in at Columbia?

Freeman:

It wasn’t so narrowly specialized, but I would say controls engineering would be the closest. It was the hot field at that time. Computers didn’t exist as a discipline. The ENIAC was just being finished and we read about computers but it wasn’t a field yet.

Nebeker:

I have read that you worked on a number of things in the first years at Sperry. Guided missile systems, what else?

Freeman:

Yes. I came into a department that was called the Special Weapons Department. That was a euphemism for guided missiles at that time. Sperry, of course, was a high-tech military electronics manufacturer. It had expanded tremendously during World War II and by ’46 the Cold War had begun. It continued to be very active in the military electronics because of this. Initially, however, it was all analog computing.

Analog and digital computers for military control systems

Nebeker:

I believe at the time that many people thought there would be a continued demand for analog computers as well as the development of digital computers. Is this true?

Freeman:

Well, in ‘48, ‘49, ‘50, digital computers were not available. I mean, they had made the ENIAC but that was it.

Nebeker:

Were there a number of projects underway?

Freeman:

Yes, but all the projects at Sperry dealt either with radar or with missile guidance. There was no effort to develop digital control computers. It was late ‘49 or early ‘50 that my boss called me over and said, “Herb, there’s something people call 'digital computers'. Why don’t you look into it?”

Nebeker:

Specifically for control systems?

Freeman:

Yes. Sperry was only interested in anti-aircraft control, guided-missile control, and similar. Then some time later it was decided that the only way to get into this field, was to build a computer ourselves. At MIT there was a project called Whirlwind I and I went up to spend a week or two during the summers of ‘50 and ’51 working on it, and then came back to Sperry to design its first digital computer.

Nebeker:

Did Sperry just want to be in touch with this new technology?

Freeman:

Yes, and that is why they had sent me to MIT. We called our computer the SPEEDAC, for Sperry Electronic Digital Automatic Computer. Computers were all called “something-DAC”.

Nebeker:

Can you tell me what it was intended for? Was this more a proof of principle?

Freeman:

We built it as a general-purpose computer, but nobody really thought that it would become a product to be marketed. In any case it was really out of Sperry’s line. Sperry was not in the commercial business as such. It was intended to get our hands into this business. Before it was even completed, we started work on other computers for applications like aircraft guidance and anti-aircraft control.

Nebeker:

Were the new products digital computers?

Freeman:

Yes. Once I started working on the SPEEDAC computer, everything turned digital. There was also some cooperation with the Hughes Aircraft Company at that time. They were developing a computer, and a man by the name of Gene Grabbe was the contact at Hughes.

Nebeker:

Were these computers actually to be in the aircraft itself?

Freeman:

Yes. These were now built with miniature tubes. By today’s standards they would still be quite large.

Nebeker:

For navigation purposes?

Freeman:

These were for navigation and bombing. To drop a bomb, you don’t just heave it overboard; you have to calculate exactly when and where to release it.

Nebeker:

I know about the analog bombing computers built during World War II. Were they doing this with a digital device?

Freeman:

Yes, by the late ‘50s. Also for navigation.

Nebeker:

Did you work on any of those systems?

Freeman:

Yes. I had the digital computer section and we started hiring and started expanding. I became a section head in ’54 for digital systems in the Special Armaments Division, headed by Myron Lockwood. By that time there were maybe 25-30 people working for the section.

Nebeker:

So this was actually a fairly fast transition from learning about a new electronic digital technology and actually building systems that were being used?

Freeman:

Yes.

Nebeker:

Were these actually being deployed by the military?

Freeman:

No. They were prototypes, some of which never made it into the field because they were canceled or new, more advanced versions were started. But they started that effort and it expanded.

Ph.D. studies; visiting professorship

Freeman:

By the way, I continued to go to school. I actually took courses for a couple of semesters at Brooklyn Poly and then went to Columbia University part-time taking courses. I started working on research for my thesis, and ultimately got my doctorate, essentially part-time. The company was quite cooperative. They let me take one or two days a week off if I needed to work on it, and then one summer I took maybe three months just to finish the thesis. I got my Dr.Eng.Sc. degree in 1956.

Nebeker:

Can you tell me what the dissertation was?

Freeman:

The title was “Multipole Control Systems.” What it meant was control systems with many inputs and outputs rather than control systems that had one input and one output. In my dissertation’s case, you have many things coming in and many things going out, and that complicated how they interacted. It was fairly theoretical, as theses tend to be.

Nebeker:

Who was your advisor?

Freeman:

Professor John Ragazzini, who was well known in the controls field. He was one of the pioneers in the field of control systems. Do you know Lotfi Zadeh?

Nebeker:

Yes.

Freeman:

Well, many of Zadeh’s early papers were actually by Ragazzini and Zadeh.

Nebeker:

Was he an influential person to you?

Freeman:

Very much so, yes, and I liked him very much. This is why when I wrote an IEEE tutorial in 1980, I dedicated it to him.

Nebeker:

Was it your idea to go to academia and get your Ph.D.?

Freeman:

Yes it was. I always had wanted to be a professor. So then in the fall of 1958 I asked the company for a leave of absence and went to MIT for one year as a visiting professor in the Servomechanisms Laboratory. That was September ‘58 to June ‘59. That summer I stayed at the Lincoln Lab, which was an MIT research facility. I spent the summer there, and then around the first of September ‘59 I returned to Sperry in the Department for Military Data Processing.

Military Data Processing, Sperry

Nebeker:

What was that data processing for?

Freeman:

All for controls-type computers that would go with military equipment. A few years earlier, maybe in ‘55 or ‘56, Sperry merged with the Remington-Rand Corporation to form Sperry-Rand. I knew Grace Hopper, who was a counterpart at Remington-Rand at the time and she came up to visit and looked at our computer. Also J. Presper Eckert from ENIAC. Of course he was with Remington-Rand. Before the merger we would sometimes talk about the possibility of having Sperry build some commercial type of data processing computer. After the merger it was clear that this did not make sense and, we henceforth concentrated strictly on computers that would be part of navigation systems, warning systems, missile-guidance systems, and things of that nature.

Nebeker:

You talked about how quickly these prototype systems, digital electronic systems, were built. Did these actually start replacing the analog systems in aircraft and missiles?

Freeman:

There was a long period, of course, from the time you have an idea about creating a piece of equipment and actually having it go in the field and work reliably.

The early computers were not reliable; so they were largely experimental. They might make one and test it, but in terms of actually using it, that really didn’t happen until transistors started coming in the late ‘50s. The transistor was invented in ‘48, but in terms of getting into computers and having systems work, it wasn’t until the late ‘50s that this really became practical. Then we built transistor computers.

So it was, the computers became smaller. The initial transistor computers used germanium transistors and weren’t very reliable. Reliable computers then began around 1959. I remember once I had to visit a navy ship for Sperry. It was a guided-missile cruiser. Apparently something didn’t work, and I had to go down with a couple of field engineers to talk to the commander and try to assure him that we could fix it. The equipment, however, just wasn’t very reliable yet at that time, and this caused the problem.

Nebeker:

The challenge at this time wasn’t so much in the design as in the actual implementation, correct?

Freeman:

Right. It would be like having a house where the bricks crumbled, but the foundation was still good. All of this improved and by the early ‘60s these problems were largely overcome.

Nebeker:

During this time you moved to the electrical engineering department at New York University?

Freeman:

Yes, I decided to make the break from Sperry. Things were changing in the company. I thought that Sperry had had a very enlightened management in the early ‘50s. There were a number of changes in management, and I just felt it was time to make a change. I had been there 12 years at that time point, and it had been my only job thus far.

New York University employment

Freeman:

I went to New York University, where in the meantime Professor Ragazzini had come from Columbia and become the dean of engineering.

Nebeker:

Was that part of your reason for going to New York?

Freeman:

Yes, and in ‘68 I became chairman of the department. I did more computer stuff while I was there. I mean I had been strictly an electrical engineer, analog computing, control systems, but had gradually shifted toward digital computers. Initially it was computer hardware, then I got interested in computer software and computer graphics.

Discrete Time Systems, 1965

Nebeker:

I wanted to ask you about your 1965 book, Discrete Time Systems, which I found very impressive. It’s interesting that it appeared just about the time that the fast Fourier transform was rediscovered.

Freeman:

It’s unfortunately slightly earlier, so others did not mention it.

Nebeker:

In the preface you talked about studying discrete-time systems because they are necessary for computer models of continuous-time systems, and, therefore, we’ve got to understand discrete-time systems. But you don’t specifically say there in the preface that you may be interested in implementing discrete-time systems. I mean, your justification in the preface is that we’re studying continuous-time systems by computers, therefore, we’ve got to understand the discrete-time systems.

Freeman:

Right. The implication was certainly that we would and by that time we were doing so, so I must just have missed it somehow.

Nebeker:

Later on in the book you talk about sampled-data control systems, which are examples of that. But it struck me, in reading your preface, that you foresaw digital processing. I mean, when I have talked to some people who were involved in the very beginnings of digital signal processing, their interest initially was simply simulating analog filters. You were already thinking of the discrete- time system.

Freeman:

Yes, my Sperry background caused this because we had already built digital systems then.

Nebeker:

That’s right. You were actually building discrete-time systems there in the ‘50s.

Freeman:

Yes. Mine was the first book, I believe, with that title and it opened up the field. Today there are many courses on discrete-time systems, and the other books written before that on the subject really described sampled-data systems. There had been some papers on the topic by Rudy Kalman and Jack Bertram, both of whom had been fellow graduate students of mine at Columbia. They were both students of Ragazzini.

Nebeker:

Yes, I was also struck by what you write in that preface: “For a number of years I have been conducting a graduate course in discrete-time systems.” That must have been one of the first courses?

Freeman:

Very likely, yes. I gave it at New York University and at the Murray Hill, NJ facility of Bell Laboratories. NYU had an arrangement with Bell Labs at that time where faculty would come out there and give courses for which the attendees would receive academic credit. It was a kind of off-campus training. They would get a degree from NYU if they took enough courses and met all the requirements, and there were a lot of students taking it. It was a big operation then.

Nebeker:

I assume that this book was well received. The Discrete Time Systems book?

Freeman:

I think so. I occasionally, even to this day, bump into somebody who mentions it.

Departmental structure and funding, NYU

Nebeker:

I was surprised, when I saw the book, that so many of these ideas that came a little bit later in signal processing were there and that you had been teaching this course for some years. How did it go at New York University? You mentioned you became head of the department in ‘68.

Freeman:

<flashmp3>339 - freeman - clip 2.mp3</flashmp3>

In ‘68, yes, and things went well. We changed the name of the department at my suggestion to Department of Electrical Engineering and Computer Science. Computer science really came out of mathematics. A professor by the name of George Forsythe at Stanford University was the first to coin the term “computer science“ breaking it out of the math department. It was a reaction to the feeling by mathematicians that working with computers just wasn’t mathematics.

Suddenly there were computer science departments springing up all over the place. NYU had an electrical engineering school and courses in computers, but computers weren’t officially recognized, and there was a danger that students who wanted to learn about computers would go to the new computer-science department that had evolved out of mathematic -- even when what they really wanted was computer engineering. The first to react to this was the University of California at Berkeley. By the way the man instrumental in doing it there, and I, quite honestly, copied his idea, was Professor Lotfi Zadeh, who had moved there from Columbia in the late ‘50s.. He arranged to have the name at Berkeley changed to “electrical engineering and computer science.” I then pushed the same idea at NYU and in ‘69 or so we did it.

Also around that time NYU created a department of computer science out of the math department downtown at the Courant Institute. The professor in charge of that department was Jack Schwartz, a mathematician. We got along well, and he didn’t object to our change of name He felt perfectly comfortable with a mathematically-oriented computer science department and an engineering-oriented computer science department.

Then unfortunately in the early ‘70s NYU got into big financial difficulties. The building program launched by President John Hester ran out of funds. Also in the early ‘70s, as a result of the Vietnam War, there was a kind of an anti-university, anti-engineering attitude. The economy was beginning to suffer a little bit. Most universities were in serious financial difficulties; NYU was heading for bankruptcy. Due to these forces the engineering department shrank. There was a feeling, “Ah, these engineers they are maybe partly responsible for the Vietnam War, probably the technology and all that.” So engineering enrollment dropped.

Brooklyn Poly at that time was called the Polytechnic Institute of Brooklyn, under President Ernst Weber, and was also having financial trouble. In fact they had already gone bankrupt and had gotten the state to feed them money to keep afloat. So now it looked like NYU was going to follow. A deal was worked out where NYU would sell the Bronx campus to the state, which would convert it to a community college to be called the Bronx Community College, for $62 million, I believe. The liberal arts college on the campus would be merged with the downtown liberal arts college, and the engineering school would be merged with Brooklyn Poly. I remember going to Albany and talking with legislators and hearing, “there are too many engineering schools as it is.”

It was interesting to see how these legislators had absolutely no understanding of what engineering was and where it fits into our society. They were mostly lawyers and accountants maybe, but definitely not technical people. So the NYU uptown campus was sold in 1973 and the idea was that the entire engineering faculty was to move to Brooklyn Poly. I looked at it, as well as a few of my friends and colleagues in the university, as a bad move. I mean, Brooklyn Poly itself had been bankrupt and we didn’t particularly like the location, so we arranged to form what was called an Applied Science Division at NYU. We couldn’t call it engineering, because engineering had been sold and merged with Brooklyn Poly.

The leader of our group was a man by the name of Antonio Ferri. He was an aeronautical engineer, but a very distinguished guy. He was Italian and had come to the US after the war. During the war he had fought with the partisans against Germany and the fascists. He was an absolutely brilliant man. I believe he is credited with being the inventor of the jet engine. He said, “Okay. We’ll start this applied science division.” At first we had some temporary quarters in the Bronx and then we moved downtown, near the rest of NYU. But this was a tough time. It was now ‘73-’74, and I was getting itchy.

Freeman:

So I started to look around. It wasn’t only because the times were tough, but I was also repelled by the political problems. Some of the rest of the faculty at NYU resisted us. Maybe they felt we were encroaching on them. So I looked around and Rensselaer Polytechnic Institute in Troy, NY made me an attractive offer. It was in Upstate New York, an area where I had grown up. I accepted their offer and went there in 1975.

Computer graphics and image processing

Nebeker:

Before we talk about your work there, I wonder if we can back up just a bit. In the late ‘60s you were already looking at some computer graphics issues?

Freeman:

I got interested earlier than that, perhaps as early as 1959 while I was at MIT. I developed what I called the chain code that you may see mentioned. In some ways it was a ridiculously simple idea but like Ohm’s Law, the simplest things often are very important.

Nebeker:

Right. I see from ‘61 on you have a lot of publications relating to geometric patterns.

Freeman:

That’s graphic data, in fact I got the King-Sun Fu Award from the International Association for Pattern Recognition in 1994, essentially for the invention of the chain code. There are people who refer to it as the Freeman Code, but officially it’s the chain code.

Nebeker:

I was particularly interested in the journal for which I know you were a founder and editor in 1970, the Journal of Computer Graphics and Image Processing. Can you tell me a little about the establishment of that?

Freeman:

Azriel Rosenfeld, a professor at the University of Maryland, and I both had research contracts from the Air Force Office of Scientific Research and so we got to know each other and collaborated on things, meetings, and so forth. Once we were at a meeting, I remember saying to him, “You know, maybe we should start a journal?” He said he had contacts with Academic Press where he was just in the process of publishing a book, and he would check with the editor whether they would be interested in it. So we launched that journal, and then invited two other people, Tom Huang and Andy Van Dam, to form the editorial board. That’s how the journal got launched, and in 1980 the name was changed to Computer Graphics, Vision and Image Processing. Then in 1990 it was changed once more. It was split into two journals. My hope had always been to keep graphics and image processing together. I felt that both deal with pictures. In image processing you process an existing picture; in computer graphics you create a picture. That is why I picked the original title, Computer Graphics and Image Processing. But by 1990 these fields had expanded so much that the publisher felt the journal should be split, and now there are two distinct journals by Academic Press.

Nebeker: 

Now when you established this in 1970 did image processing then mean what it does today?

Freeman:

Both computer graphics and image processing are sometimes interpreted differently. There were people who considered computer graphics to include image processing, and there were people who considered image processing to include computer graphics. Therefore these are not black-and-white divisions. I would say it’s still pretty much the same today. In those days computer graphics was largely Computer Aided Design, CAD. The art world wasn’t interested in it yet.

Today if you go to a computer graphics conference, 90 percent of the people are artists, people from multi-media and the entertainment industry. It has largely become an entertainment industry tool today.

Nebeker:

I’m also curious about what is encompassed by the term “image processing?”

Freeman:

The military, of course, was always interested in it for aerial photographs, satellite photographs, and intelligence.

Nebeker:

But is this specifically the computer processing of imaging?

Freeman:

If the image was taken photographically, it would be scanned, converted to digital signals, and then processed digitally. In many cases, especially later, the data image may have been obtained digitally to begin with.

Nebeker:

So the articles that were included in that journal dealing with image processing would all be computer processing of images?

Freeman:

Yes.

Nebeker:

And how did it go for this journal?

Freeman:

I don’t know, but Academic Press seemed happy with it. And just this past year I resigned from the editorial board.

Nebeker:

That’s quite a few years.

Freeman:

I was on the editorial board of that journal as well as on the board of the Journal of Pattern Recognition, which was formed about the same time. Pattern Recognition is closely related but with a different emphasis. In the late ‘70s K.S. Fu and I launched a new IEEE Transaction, PAMI, the IEEE Transactions on Pattern Analysis and Machine Intelligence, which exists to this day. I was on the editorial board of that journal for many years as well.

Nebeker:

What other journals, if any, were there in computer graphics and image processing back in 1970?

Freeman:

Well, I don’t think there were any in 1970. Some years later the Computer Society launched Computers and Applications, a more general journal, somewhat like Spectrum but concentrated on the computer field.

Nebeker:

I’m just wondering if this qualifies as being the first scholarly journal in either of those fields, computer graphics and image processing?

Freeman:

It certainly is one of the first. Whether it is the first, I don’t know, there could have been others.

Nebeker:

And it was because you thought there needed to be an avenue for publications in this area?

Freeman:

Yes, and we immediately got good papers published in them. Of course pattern recognition, image processing, computer vision, computer graphics, -- they are all related. In many cases they actually overlap. One could say that computer vision is a combination of image processing and pattern recognition. In image processing sometimes all you want to do is take an image and sharpen it. So you put it in the computer and process it. But you haven’t really interpreted what’s in the image.

That’s where pattern recognition comes in. People then use computer vision as combining the two, because like human vision you look at something and you recognize it.

Rensselaer Polytechnic Institute, image processing lab

Nebeker:

Yes. Now at Rensselaer you were the first director I believe of the image-processing laboratory?

Freeman:

Yes.

Nebeker:

Can you tell me about that establishment?

Freeman:

Yes, you know it’s not a big laboratory in any sense. But we got students and research support in image processing.

Nebeker:

Did that include computer graphics?

Freeman:

Yes. Except that RPI started a separate computer graphics lab under Michael Wozny. He’s an active IEEE person. That initially started out more CAD-oriented but also became quite successful and became called the Computer Graphics Laboratory. (The actual title may have been somewhat different.)

We concentrated more on image processing and pattern recognition for computer vision, and that went well. There was also a push to change the name of the department. When I arrived in 1975, it was called the Electrical and Systems Engineering department, and I promoted putting the word “computer” into it, so that it became the Electrical, Computer and Systems Engineering department.

In many cases the faculty, especially older faculty, offered resistance. If they are not in the computer field, they may feel, “it’s going to diminish my role a little bit.” I tried to convince them that this is not so, that it would strengthen the department, “the tide will lift all boats,” so to speak. I think that’s been the case. We have a curriculum here at Rutgers in computer engineering, and I very much wanted to keep electromagnetic theory as a required course in that curriculum. But just this past year I got out-voted and it was taken out. Still, I felt it was something important to keep for the education of computer engineers.

International Association for Pattern Recognition

Nebeker:

I wanted to ask also about 1978-80 when you were president of the International Association for Pattern Recognition.

Freeman:

<flashmp3>339 - freeman - clip 3.mp3</flashmp3>

Yes, I helped found that. Again, King-Sun Fu was very helpful in it. He and I were very good friends there’s a picture of him here on the wall. It started out just as a conference committee. The first conference was held in 1973 in Washington, D.C. Azriel Rosenfeld was also involved, and was probably the chairman of that conference. The second one was held in Copenhagen, Denmark in ‘74.

I was program chairman of the International Computer Congress that year held in Stockholm, Sweden. That was of course a very big thing. It was sponsored by IFIP, the International Federation for Information Processing. The IEEE has been involved with IFIP, and I suppose it still is. At the time there also was AFIPS, an American organization but that got shut down.

In Copenhagen we had the second conference called the Joint International Conference on Pattern Recognition. The IEEE published the proceedings. After the Copenhagen conference I pushed very much that we form a permanent organization. I had a lot of experience in this. I had been involved in IFIP since ‘62. That’s why I became the program chairman in ‘74, but it was a long history of moving up to that.

In '65 IFIP had a congress in New York City. I was in charge of the local arrangement committee for that congress. In ‘68 it was Edinburg, UK and I was on the U.S. committee to organize that. In ‘71 it was in Lubliana, now Slovenia but then Yugoslavia, and I chaired the U.S. committee. And then in ‘74 I became the program chairman for the ‘74 meeting. Then in the late ‘70s I was even the US delegate to IFIP.

Nebeker:

It sounds like you have a considerable amount of international experience from your work.

Freeman:

Yes, and I found IFIP very useful in making international contacts. It was wonderful that way, and of course that was what it was intended to do. Is the name Isaac Auerbach familiar to you?

Nebeker:

Yes.

Freeman:

He unfortunately died about 3-4 years ago. He was really the founder of IFIP and single-mindedly kept it going . He lived in Philadelphia. -- Well we got IAPR [International Association for Pattern Recognition] going. I arranged to incorporate it as a nonprofit organization in the State of New York. K.S. Fu became the first president and I was the first treasurer.

Nebeker:

Did you establish a journal at the IAPR?

Freeman:

We thought about it but felt that there were enough journals so we did not create a journal at first. We did create a newsletter, and that exists to this day. A journal then was separately created, Pattern Recognition Letters. It was published by North Holland, which is now called Elsevier. IAPR, like IFIP, was not going to be an organization of individual members but an umbrella organization. The IEEE is the U.S. member in IAPR.

IEEE societies

Nebeker:

I’m curious about the division of labor between the Signal Processing Society and the Computer Society in this area of image processing. How do you see that division?

Freeman:

I can’t respond to that very well. I have had very little contact with the Signal Processing Society. I never attended any of their meetings. But that was because my personal interest was never so much in signal processing. My interest was more in pattern recognition with focus on content rather than form.

Nebeker:

Are you talking about how you encode the image for transmission, say?

Freeman:

That’s right. I was never interested in encoding for transmission or compression.

Nebeker:

These are typical things that signal processing people do.

Freeman:

Absolutely. Now there are others who do different things, but I think in general that’s true of IAPR. It’s really quite disjoint from the signal processing committee.

Definition of image processing, computer science, and computer engineering fields

Nebeker:

Yes. I think it may be difficult for outsiders to understand the vagueness of that term “image processing.” I mean it can be the physical signal, but it can also be more the analysis and use of images in computer. Can you talk more about this distinction? Do you see this as being wasteful overlap or a definitional problem in the field?

Freeman:

I think you have to let nature take its course in that. Yes, I am a little saddened by the conflict over the computer science and computer engineering separation. I don’t think there ever has been a conflict between signal processing and these other focuses. I mean that there are professionally oriented communities, and the people in them mutually respect each other and collaborate where it appears desirable to do so. But in computer science/computer engineering within the university environment there is more of a competition for resources and for students, all of which creates real division. That is usually where the conflict is, and that is unfortunate.

In my opinion, the computer field is an industrial field just like chemical engineering, mechanical engineering, civil engineering, electrical engineering, and computer engineering. There is after all an enormous industry out there that’s oriented toward computers. Science does not directly serve industry. It serves it indirectly. Don’t get me wrong, I don’t want to belittle science. There is a place for people to work on the theoretical aspects of computing. They should be closely allied to mathematics, but they are not engineers and they shouldn’t call themselves engineers. Just as engineers shouldn’t call themselves scientists and then do engineering. It misleads students, it misleads the world at large.

I can give you an example. I know somebody very well, a bright young man who got his doctorate in computer science at the University of Illinois. He did fine work there and was hired by Hewlett-Packard for their research department. Subsequently, he told me that they were sending him back to school for additional training. I mean he was hired already, but then they said, “You know, we’ve got to send you back to school.” It turned out he had never had a course in physics in his lifetime, as well as a few other things. I mean, I don’t even know how much mathematics he had had. He was very, very good in computer science but had come through a computer science program that was basically a liberal arts program. Hewlett-Packard sent him back to school for one year—after he already had his Ph.D. – to catch up on what he lacked.

Nebeker:

So even a big company can be misled by the term?

Freeman:

Not necessarily, they might still have hired him because he was a bright guy and so forth, okay? But that would never happen with an engineer. There is no such thing as an engineer who doesn’t have at least two or three years of physics as well as a lot of other things. Therefore, engineering is an important discipline. Engineers are the builders of society but here in the US we denigrate it, and I think in the long run this can have very bad repercussions.

You can see evidence of it if you look at our graduate student body. You will see that an extremely high percentage today comes from other countries. Now it’s wonderful that they want to come here and study with us. The question is, where are the American students? Why are they not coming to engineering? I mean the good students, in particular.

Map data processing

Nebeker:

I did want to ask you about another area we haven’t touched on and in which you’ve done a good deal of work, and that’s map data processing. I suppose this can be thought of as a part of image processing.

Freeman:

Yes, it is to some extent. It involves pattern recognition and it involves computer graphics. Very much these things tied together.

Nebeker:

Has this area been important commercially? Because map making is such an important activity?

Freeman:

Yes.

Nebeker:

So is there a good connection between the more theoretical research work and the industry of producing maps?

Freeman:

Well, that connection is just coming about. In 1977 I organized a conference under the sponsorship of the IEEE Computer Society on the subject of pattern recognition and image processing to be held in Troy, New York. It then became a series, and has been held every two years since.

At that conference I met a geographer who said to me “Herb, today in making maps, we do everything with computers except for two things which we can’t do. We have tried, we have had a lot of people work on it, but we can’t do it.” So I asked him to tell me what they were, and one of them was the automatic labeling of maps. I said, “Well, why is this complicated?” He explained it to me and I got interested and said to myself, “this sounds like a good research topic.” Sometime later I had a new doctoral student coming by, and I said, “let’s make this your doctoral research.” He and I then worked on it. I got some government support for it, I believe, from the National Science Foundation, and his thesis was very successful. It got national attention. He presented a paper in Ottawa at a conference and the audience applauded, which at a scientific meeting is very unusual. That was the first Ph.D. thesis. There was a second one a few years later and also a number of Master’s theses. We’re doing this kind of work still today.

Nebeker:

Were these programs actually used by mapmakers?

Freeman:

<flashmp3>339 - freeman - clip 4.mp3</flashmp3>

Yes. To this day, most maps are labeled manually. We’ve had some big projects here at Rutgers in the last few years. One was with the U.S. Agriculture department, and that was very successful. We delivered the software to them and they are using it. They had come to me and said, “We make these soil survey maps. Can you help us automate the placement of the labels?” The maps are about the size of this table, and there is a map for every area in the U.S. If you are a land developer, before you buy a tract of land, and certainly before you start building on it, you had better find out what kind of soil there is by consulting a soil map.

Before we provided them with our software, they literally labelled their maps by hand. That is, they made a list of all the labels, sent it to a printer who printed it on a sheet of clear plastic with adhesive backing, and then the cartographer would peel the labels off one by one and paste them in their proper place. Imagine how tedious this can be, especially when a map could have 2500 or more labels! They also told us that they had given contracts to big companies but that all had failed.

We tackled the job. It took us some time and eventually got it to work. We even got two patents on the process. The software was delivered and it’s in use. It was a big success story. The Census Bureau then came to us and said, “We’re gearing up for the 2000 census. We have to make 1.5 million different maps for it. Can you help us label them?” And we were successful in that effort as well. The 2000 census maps will all be labeled with our software.

Nebeker:

Is that right? It’s very interesting how you explain the problem to a layman, I’ve looked at some of your papers and it is hard to appreciate that the problem can be both so difficult and also so important.

Freeman:

The map labeling problem is a difficult problem. There are many things that humans do easily and yet are extraordinarily difficult to computerize. For example, the easiest thing for you is to recognize me the next time you see me. You didn’t take any special course in this. It’s so easy for you. Okay? Yet try to get a computer to do that!

Nebeker:

One would think image processing and recognition would be completely straightforward for the computer since humans do it with relative ease?

Freeman:

Maps are pictures of nature; every map is different and very intricate. You must never have text overlap other text. There must be no ambiguity. It must be clear which label goes with which geographic feature. Then there are subtle things. For example, say, there’s a village on the border between New Jersey and Pennsylvania. It’s in New Jersey. There’s no other village around for miles. You might think you can put the name anywhere near it. A cartographer will say, “no, be careful. If the village is in New Jersey, make sure that you put the name in New Jersey. If the village is in Pennsylvania, the name’s got to be in Pennsylvania.” You’d say, “Why? There’s no ambiguity.

At the moment as you look at the map it’s clear. But when you put the map away, and a few days later you think of that village, if the name was placed in Pennsylvania, you’d think the village was in Pennsylvania. There are many situations like that, and that’s where you can see this problem gets complicated.

Nebeker:

Let’s discuss another book that you edited, based on a workshop on computer architectures for spatially distributed data. I’m always interested in the connections between people who work on the algorithm side and people who work on the chip design, the hardware side. Was the objective of this workshop to bring these communities together?

Computer architectures for spatially distributed data

Freeman:

Yes. In 1983 image processing was computationally intensive, because a good image, say, for military purposes, had to be of high resolution. Otherwise it wouldn’t be any good. As a result, there were lots of pixels and a lot of processing, and the standard computers at that time were very slow. Therefore, a lot of effort was placed on developing specialized computer architectures to work with such data. The data is obviously spatially organized, that is, you have an array of maybe 4000 pixels by 4000 pixels. Consequently a computer that had 4000 by 4000 little processors might be the best way to work with it. So this book was about that kind of thing. Today we still have Crays and other big computers, but even the more general-purpose computers are now better for dealing with such data.

Nebeker: 

Yes. But I know that this happens in other types of signal processing, that both the algorithm designers are aware of what the capabilities of the chip are and the chip designers will also be aware of what algorithms are going to be used there. Do you think there has been good communications so far in the areas you’ve worked between the two sides?

Freeman:

Yes. It comes about through research contracts and contacts at research oriented meetings, usually ones sponsored by the Department of Defense. Every two years there is an image processing conference here in the US. People from both camps would be there. On the research level usually cooperation is very good because it’s in everybody’s interest to get together and exchange ideas. There is also a conference on computer architectures for image processing that the Computer Society sponsors.

Rutgers Center for Computer Aids for Industrial Productivity

Nebeker:

The final topic I want to ask about, is here at Rutgers, the Center for Computer Aids for Industrial Productivity. You were the first director?

Freeman:

Yes.

Nebeker:

From 1985 to 1990. Would you just briefly tell me how that came about?

Freeman:

I didn’t create the Center. At that time, Dr. Pond, who is the executive vice president, and Dr. Blaustein, the former president, worked with Governor Keane to start it. Keane said he was very pro-education, I didn’t hear him make the statement but heard that he wanted Rutgers University to be for New Jersey what the University of Michigan is for the State of Michigan. He envisioned a real flagship that guided the intellectual and scientific future of the state.

Together with then the administration here at Rutgers, they arranged for the state to create a Commission for Science and Technology and that commission got money from the state to set up research centers. A proposal was made—I was not involved in it—for something in the computer area. And the title was going to be CAIP, the Center for Computer Aids for Industrial Productivity. Today they would probably choose a different title. They then looked for a director for this center. Saul Amarel at that time was chairman of the computer science department. He’s an electrical engineer and a fellow graduate student of mine from Columbia University.

Nebeker:

The Columbia Mafia.

Freeman:

Yes. He called me at RPI and said, “Herb, we have a position here at Rutgers that you might be interested in.” He actually said, “Well, there are actually two positions – head of the CAIP Center and one as professor to create a computer engineering program.” I had been active in that area and had published an NSF-sponsored research report on computer engineering a few years earlier. The report had gotten a lot of attention and it helped also to publicize the word “computer engineering.”

To make a long story short, I came down and interviewed. They asked me which of the two positions I preferred. I sensed there was some political tension over the CAIP center as to exactly who would control it among different departments and faculties. So I said, “Well, I’ll opt for the computer engineering program,” which was a more narrowly defined kind of thing. That was agreed and I accepted.

A couple of weeks later Dr. Pond called me on the phone, still at RPI, and said, “Herb, we haven’t been able to find anybody for the CAIP center directorship. Will you take that also?” And, maybe foolishly, I said yes. So I came in taking on both positions.

Nebeker:

How did it go for you?

Freeman:

Okay. I think the beginning was a little difficult. We were starting from scratch. But by the time I left it five years later, we had 30 strong member organizations. There was a book written in the ‘60s by a man named White on industrial research in the United States. I don’t have the exact name, but it’s a well-known book. At one point he says in the book that there are really only three really top industrial research centers: Bell Laboratories, the RCA Research Laboratory, and the General Electric Research Laboratory. A fourth one, the IBM Research Lab, hadn’t quite yet reached that stature. I mention this because in my report at the annual meeting of the CAIP center in 1990 I was able to announce that all four of these organizations were members of CAIP. Unfortunately I think that it’s not true anymore today, but the industry has changed so much. But I am still in charge of the computer-engineering program, which is part of the Electrical and Computer Engineering department.

Nebeker:

Thank you for the interview.

Interview addendum, 2007

The interview was conducted in June 1998. The following paragraphs, provided by Herbert Freeman, updates the interview to include Freeman's activities from 1997-2007.

MapText, Inc.

Freeman:

In April 1997 I founded a company called MapText, Inc. We had been developing software for the US Census Bureau that would automatically label the features (counties, streets, rivers, towns, etc.) that would appear on the Census Bureau’s maps. Automatic labeling of map features was a task long believed to be beyond a computer’s capability. But we demonstrated that it was feasible in a research program at Rensselaer Polytechnic Institute during the late 1970s and early 1980s. The Census Bureau liked the software and intended to use it for the Census 2000, when they expected to produce some 20 million fully labeled maps. They wanted a commercial organization, not a professor with some graduate students, to provide support for the software. Existing commercial organizations expressed no interest, and thus the only choice left was to form a new company, which was done in 1997. Initially the company devoted almost its entire effort to assure success of its software for the Census 2000, but eventually other organizations also expressed interest and emphasis was placed on developing map-labeling software that would be of broad interest worldwide.

Since work in the company became quite demanding of my time, I felt I had to resign from the university. I informed the university of this in 2000 and retired as of January 2001. I then could devote my full energies to MapText. The company prospered, with contracts from both governmental and map-making organizations worldwide. Then as of November 30, 2005, I sold the company to Lufthansa Systems Inc. Their primary interest was in automatic labeling of aeronautical charts. There was a four-month period during which I helped the new management, and as of March 30, 2006, I fully retired, nearly 60 years after I had graduated from Union College with a BSEE degree. Since that time I have been writing my memoir and also some papers I had long intended to write.