Oral-History:Ruzena Bajcsy (2010)

About Ruzena Bajcsy

Ruzena Bajcsy was born in 1933 in Bratislava, Slovakia. Following graduation from the school of electrical engineering at the Slovak Technical School in 1957, she entered the industry for five years. She later achieved a Ph.D. in electrical engineering in 1967 and a Ph.D. in computer science in 1972 from Stanford. While previous research centered on robotics and automation, her current focus is on artificial intelligence; biosystems and computational biology; control, intelligent systems, and robotics; graphics and human-computer interaction, computer vision; and security. For her research and work in the field, she has been recognized with several awards, including the 2013 IEEE Robotics and Automation Award, involvement on numerous advisory boards and committees, and the authoring of more than 225 articles in journal and conference proceedings, 25 book chapters and 66 technical reports. She is currently a professor of Electrical Engineering and Computer Science, as well as Director Emerita of the Center for Information Technology Research in the Interest of Society (CITRUS) at the University of California, Berkeley.

About the Interview

RUZENA BAJCSY: An Interview Conducted by Peter Asaro with Selma Šabanović, IEEE History Center, 17 November 2010.

Interview #664 for Indiana University and IEEE History Center, the Institute of Electrical and Electronic Engineers, Inc.

Copyright Statement

This manuscript is being made available for research purposes only. All literary rights in the manuscript, including the right to publish, are reserved to Indiana University and to the IEEE History Center. No part of the manuscript may be quoted for publication without the written permission the Director of IEEE History Center.

Request for permission to quote for publication should be addressed to the IEEE History Center Oral History Program, IEEE History Center at Stevens Institute of Technology, Castle Point on Hudson, Hoboken, NJ 07030 USA or ieee-history@ieee.org. It should include identification of the specific passages to be quoted, anticipated use of the passages, and identification of the user. Inquiries concerning the original video recording should be sent to Professor Selma Sabanovic, selmas@indiana.edu.

It is recommended that this oral history be cited as follows:

Ruzena Bajcsy, an oral history conducted in 2010 by Peter Asaro with Selma Šabanović, Indiana University, Bloomington Indiana, for Indiana University and the IEEE.

Interview

INTERVIEWEE: Ruzena Bajcsy
INTERVIEWER: Peter Asaro with Selma Šabanović
DATE: 17 November 2010
PLACE: Berkeley, CA

Early Life and Education

Q:

…by asking you to tell us where you were born and where you grew up.

Ruzena Bajcsy:

I was born in Bratislava, which is Slovakia, used to be Czechoslovakia, and what else?

Q:

And you grew up there and you went to undergraduate there?

Ruzena Bajcsy:

And I grew up there and I went to the undergraduate school of electrical engineering at the Slovak Technical University. I graduated in 1957, and then I worked in industry for five years, and then I joined the university computing center, which bought the first computer in Slovakia from Russia. The computer was an electronic cube computer called Ural II, and I worked first as a maintenance engineer, and then maybe three years later or so I became an assistant professor there in the newly established computer-science department. In meantime, I also worked on my Ph.D. thesis, which I defended in 1967. And then I got an opportunity to come to Stanford and I arrived to America in October 1967, and initially I was going to stay only for one year, but then I got the opportunity to enroll as a graduate student, and since my degree was in electrical engineering I realized very quickly that I didn’t know what computer science was. So, I enrolled as a Ph.D. student in computer science and I got my second Ph.D. in compter science in 1972 from Stanford. And then I took a job as an assistant professor at University of Pennsylvania in Philadelphia in the newly established computer-science department.

Q:

So, what was your thesis on for your two degrees?

Ruzena Bajcsy:

Well, in electrical engineering my thesis was on machine learning. I developed a model of – basically was a feedback system, teacher-student kind of learning, probabilistic learning. And then at Stanford my thesis was on computer vision in texture analysis – was first work on texture analysis.

Q:

And who were your advisers?

Ruzena Bajcsy:

John McCarthy.

Q:

And then in Slovakia?

Ruzena Bajcsy:

In Slovakia you don’t have really advisers. You basically work and then you have a committee, and..

Research at Stanford

Q:

How did you get the opportunity to come to Stanford? How did that happen?

Ruzena Bajcsy:

That’s an interesting story. My boss, Professor Gvozdjak, who was the chair of Slovak computer-science department, got the opportunity to go to England to look at the computers in Great Britain. I believe he went to – I forgot, but he looked at the hardware, the new architectures in England. I’m not sure whether it was in Edinburgh or – no, it was in Manchester. That’s where the first computer – the British first computer was built. And he met there John McCarthy, and John suggested that he would be interested to take somebody from Slovakia to study here at Stanford, and so when Professor Gvozdjak came back, he told about this opportunity, and I jumped on it. And since I was the first woman who got a Ph.D. in electrical engineering in Slovakia, that gave me a little edge in the competition. And so I was selected to go to Stanford.

Q:

And what were the differences? So when you moved from Slovakia to Stanford, were there differences in..

Ruzena Bajcsy:

It was tremendously different, yes. First of all, you realize that in Europe in general the Ph.D. people don’t take courses. They just work on their independent research, and while here people have to pass examinations and take courses, so that was really a big shock to me, because also during the undergraduate studies in Slovakia the examinations were mostly oral. I mean, in mathematics and physics you had the combination first, solving some written problems, assignments, and then you went for an oral examination. In America everything was a written examination and it was time-constrained, and I wasn’t used to that. So, my first exam was a failure, but Professor McCarthy was very kind to me and he supported me, and so I got a second chance, which I finally passed, and then my research was okay, so that’s how I made it. But the beginnings were very rough, because my English was not so good. I was not used to the examination system, and so it was – and I was competing with very highly trained – in mathematics, in particular, highly trained people. And I also was a little older, so I was competing with ten years younger cohorts and they were much faster than I was, so it was rough, but I <phone rings> persevered. Never mind. I’m not going to take the call.

Q:

Were there differences between the kind of programming languages that you were using and..

Ruzena Bajcsy:

Well, in Slovakia I was coding in hexadecimal system. I mean, there was not even an assembly language on this computer that I – well, here I came and we were using IBM. I don’t know whether it was 360 or the precursor of IBM. We used punch cards, and so every time you made an error you had to rate again, so it was a very different thing. And the programming languages that McCarthy was pushing was Lisp, which was a very different language than arithmetically based language like ALGOL or Fortran, so I had to learn very quickly the recursion and the Lisp language, so it was very, very hard.

Q:

And what kind of vision systems were you using for the texture navigation?

Ruzena Bajcsy:

Well, in those days there were no CCD cameras, so it was a regular camera like yours and then attached was a digitizer, which provided you digital images, and then you worked offline on these digital images. And the challenge there was that first of all the memory in computers were very small. We worked on small images, so the resolution was not very good, worked on 64 by 64 pixels or at best 128 by 128 pixels, so the resolution was rough, and in texture you really need a higher resolution in order to capture the local properties, so that was a challenge. Technology just wasn’t there. Everything had to be made in the lab. We had to build digitizers and deal with swapping the memory to disk all the time, because the memory was typically 64K or so or even – then later on I worked on PDP-11, which had a 32K memory, the cache, so it was very different time, and therefore the approaches you took were quite different than today people use millions of pictures and do statistical analysis, very different.

Q:

Do you think that’s the biggest change you’ve seen in the technology?

Ruzena Bajcsy:

Well, no. There are many other changes. I mean, of course the speed and the memory size of the computer is phenomenally different, I mean, but the parallel isn’t that the distributed computing is also tremendously big difference and the display systems. At that time we used – when I worked at Stanford we didn’t have even a raster display. It was all just vector displays that displayed only the contours. At the end of my stay at Stanford we got the first raster display, which allowed you to display all the pixels, but in the beginning it was all just displaying contours.

Q:

Were you working on any particular applications or..

Ruzena Bajcsy:

No. That was one thing that the lab had absolutely freedom, although it was supported by the military, the DARPA – at that time was ARPA, I guess, or DARPA. I don’t remember. It changed names, but any case we were given complete freedom to do – really explore just the basic stuff. Basic research was the key.

Research and Robotics at UPenn

Q:

And so when you went to UPenn after Stanford, what kind of projects did you start working on?

Ruzena Bajcsy:

Well, that’s a different story. When I moved to Penn there was nothing, and so I had to build everything from scratch. My first master student, Adam Snyder, was an electrical engineer, and he and I built the digitizer, because we had to buy these analog cameras – camera, one camera – and then build a digitizer. And then I got some NSF initial grants that allowed me to buy a raster display so that we can visualize. The department had PDP-11, so I was working on that computer, and, as I said, it had just a very small memory, and so we were swapping things back and forth. I started to work on – I basically wanted to continue my Ph.D. work, and one of the first things that I started to look at was texture gradient and how can you interpret the three-dimensional information from the two-dimensional projection of the texture gradient. So that was the first Ph.D. with Larry Lieberman, right, was my first Ph.D. student, who then went to IBM research and worked on – had a group in robotics, because I was always interested in the three-dimensional interpretation of the visual information from the very beginning, and I always was interested in vision for, not just vision per se but what is vision for. I thought that it was a good way of testing your algorithms, if you can find the interpretation where you are or can you grasp things or move around? I mean, the robotics gives you a very good testing of your sensory processing, so that’s what I focused on. And then I think it was maybe ’73 or ’74 a man by name Britton Chance, who just passed away yesterday, 97 years old, who was a very prominent biophysicist, he invited me to look at some of these X-ray images from rat brains, because they were really interested in automating the image processing of these medical images. So he brought me into this group, and I found it very interesting, and so that started my career in medical image processing. I worked with him or for him for about six months, but then he was a very strong personality and my joke was you worked for him or you didn’t, and so I quit because I didn’t want to work for him. But then I continued with other people in the medical school at Penn on medical image processing, and I did some nice work there.

Q:

What was the first robotics system you worked on?

Ruzena Bajcsy:

Well, the first robotics system was when I got money to purchase a PUMA robot, which was one of the first robots available for academic labs. And around ’76, ’77, I started to connect the manipulation with the camera work, and grasping was one of the – and of course in those days they had only two-finger grasping, so I initiated with the Penn mechanical-engineering department to design a three-fingered robot, and so we were very much ahead of time in this regard. Around that time I also connected with a French laboratory in Toulouse, which had a rubber sensitive pressure sensor, and as part of the collaboration they made me what we called a “French finger,” which had sensors, so we were able to do this kind of a tactile connection. And so one of my first Ph.D. students – well, he wasn’t my first, but I don’t know, third or so. I think it was third or fourth. Peter Allen, who’s a professor now in Columbia University, he did the first work of this interaction of tactile and visual information, and so that was that.

Q:

Who did you work with on designing the three-fingered robot and also in this French lab?

Ruzena Bajcsy:

With the three-fingered hand? This was – well, I forget the name of the professor in the mechanical-engineering department. He passed away too. I kind of outlive all of my collaborators, but I think the student who designed it was Abramson, if I remember, three-fingered hand with a palm, and then we put some tactile pressure sensors on that, so that allow us to find where you are connecting. And actually, Ken Goldberg, who is here a professor, did his master degree with me on using these tactile sensors for recognition purposes. So, I also during that time connected with psychologists, who were prominent in this tactile perception. One of them was Susan Letterman, who is an outstanding professor-scientist in tactile perception at Queens University in Canada. And then she connected me with another professor, Roberta Klatzky, who is a professor at CMU in the psychology department, and we had a very fruitful collaboration. I learned quite a bit from these two people on how people perceive that – for example, people have certain procedures, motoric procedures when they want to find out how hard something is or what is the surface texture, and so there are these modules that people – procedures that people use for exploring. They called it “exploratory procedures.”

Q:

And so did you end up applying similar procedures to the robots?

Ruzena Bajcsy:

Absolutely implemented them, yes.

Q:

Were there any particular challenges in that?

Ruzena Bajcsy:

Well, the challenge was that the human skin has a very nice and high resolution, and my tactile sensors were basically on and off, were not very highly resolute, so we had to supplement it by multiple measurements, and so I need to really – and of course you could really only observe behaviors. You couldn’t go into what went on in your brain, so we were inferring more the matching of the computational models with the psychological – on the behavioral level. That’s all what we were able to do, but I was really impressed by how carefully these psychologists do their experiments, because people have so many capabilities and how then you want to measure one out of 100 different possibilities, how you have to isolate everything else in order to measure the one that you want to measure. So it was really an educational collaboration. We had a joint ONR Navy grant that allowed us to do that. Then unfortunately we couldn’t continue. Well, our funding expired and we didn’t continue, but I am still in touch with them, I mean.

Q:

Are there other psychological theories of vision and perception that were important to your work?

Ruzena Bajcsy:

Yes, J.J. Gibson, absolutely. J.J. Gibson was my hero, because he really had this ecological affordances approach to psychology, and that very much fitted with my way of looking at the world, so he inspired me by active perception, where he recognized this we don’t only see but we look and we don’t touch only but we feel, that there were some control strategies. And that fit me extremely well, because from Czechoslovakia I had my control-engineering background, so I was able to connect this control approach with the signal-processing approach that was part of the sensory perception.

Cybernetics in Czechoslovakia

Q:

Some of your early publications were in Kybernetika – was Cybernetics?

Ruzena Bajcsy:

That’s correct.

Q:

So what was Cybernetics like in Czechoslovakia?

Ruzena Bajcsy:

Well, it’s an interesting question, because, as you know, Cybernetics was started actually before the Second World War by people like Ashby, who were – and then later on Wiener, and these people were in Vienna before the Second World War, and they were very much interested in using the analog feedback system theory, control theory for modeling some of the psychological behavior. Ashby built this first mice <break in recording> kind of running around the maze, and that influenced me quite a bit, coming from this control-theory background to interested in – I always was interested in these biological systems, how they work, and so that connection was very natural to me from Czechoslovakia. And of course, as you know, the Russians have a long history in control theory. Pontryagin and these people in Moscow were very prominent in the feedback system, the continuous feedback system. So then when I came to America, John McCarthy and Marvin Minsky, who were originally students at Princeton and studied under Church, who was big on the logic – they were basically logicians, plus they were also influenced by the digital – the switching theory, and the digital technology was coming slowly around.

And so John really had this belief that the – what is really important is to think about representations as opposed to this feedback system. He really was interested in what do we store from our experiences, and he also believed that what we store is not a continuous signal but some kind of an extraction of this symbolic representation. And so when I came to work under him in the Artificial Intelligence Laboratory, the emphasis was in this symbolic representation. Actually, I don’t know whether you know this, but in ’64 or maybe ’63 or maybe even before, there was a famous meeting that he and Shannon organized, where they put up first this hypothesis on representation, because, as you know, Shannon was interested in the information that – the channel theory.

And then he went to Washington and wanted to convince the government to support the research, and he told me, personally, that he introduced a word, artificial intelligence, in contrast to cybernetics as an argument that what he was proposing was very different than what the cyberneticians were doing. Now there was a parallel of cybernetics here by this guy Rosenblatt of neural networks. They all were intrigued by what we carry in our head. And how to use the current know how of technology or mathematics and technology to model what’s happening in our brain. And that’s – so John and Minsky and then later on Simon and Newell, they really adhered to this hypothesis that it’s more important the representation as opposed to the feedback, and that this representation is symbolic. So that’s what – while the Eastern Europeans, including the Russians, were adhering more to the cybernetics, the analog feedback – now, remember that in this country also there was the control theory group that was still pushing the feedback systems and estimations of systems and so forth.

But McCarthy made it very clear that he didn’t want to do that. So he really wanted to be different. And, initially, people were very excited that this is it. This is really is going to revolutionize the understanding of intelligence. I think we came long way. And I think now there is much more effort to combine this analog and – or continual systems with the digital systems under the name of hybrid systems. And, as you know, Mother Nature doesn’t care how we call it. And I mean the mathematics is really an abstraction, it’s a model. Okay? Mother Nature does its own things, including the biological Mother Nature. And we still don’t know what we carry here. Right? But in the meantime, as engineers we want to build some useful systems. And to me it’s rather clear that some extraction is useful, but you have some kind of a hierarchy where you have the initial signal and then you compress and compress and compress, and clearly symbolic information – the words, is most compact representation that you can think of. Right? But, it’s also ambiguous unless you put underneath lot of the semantic interpretation and the contextual interpretation because the same word can have different interpretation. My classical example I like to use is what is large, what is small? It’s very irrelevant, right? Relative, not relevant. Relative, yeah. And so forth and so on. So you can have many interpretation depending on the context and –

Q:

So were you at any of these meetings that you talked about with McCarthy and –? Were you present and –?

Ruzena Bajcsy:

Oh, yeah, yeah, yeah, yeah, yeah. He was a good friend. I mean I really cherish our relationship, and it’s very unfortunate he’s not so well right now. But I have lot of affection for him. He really facilitated, enabled who I am, absolutely. And I will be indebted for that forever.

Q:

So what kind of machine learning algorithms were you using before you came to the States?

Ruzena Bajcsy:

Oh, okay. It was a feedback system. And it was modeled after some psychologist, Bush and Mosteller wrote the very famous book on the subject. And it was a basically a first order difference equation put into a matrix formulation of transitional probabilities and had coefficients of how fast can you learn, and what the saturation is. And that was motivated by these psychologists. And I just put it into this mathematical formulation.

Psychological Influences

Q:

How did you come into contact with all the work by psychologists very early on?

Ruzena Bajcsy:

Books. Just books. Yeah.

Q:

But how did you decide to actually go to psychology from all of the –?

Ruzena Bajcsy:

Well, I was really always very intrigued by – well, first of all I always loved mathematics. And in a different world I would have been a mathematician probably. But under the Communist regime, I looked for engineering, which had the most mathematics because the mathematics was very colored by the Communist philosophy. So – and they needed engineers. They were more tolerant with your philosophical views. So I felt more – that my future might be better served if I’m trained as an electrical engineer. But I was always interested in how people behave, how people do things. And so it’s very intriguing. And it’s still intriguing. I mean after all – I mean look what kind of robots are we building, right? I mean I have a colleague here, Pieter Abbeel trying to have a robot to fold shirts, and stuff like that. I mean we are aiming for more and more sophisticated activities, mimicking people.

Actually this was the original ambition of John McCarthy, as well. And it’s fascinating how it proceeded that, for example, when I was there he was pushing very hard for these chess programs. Okay? And it turns out that actually in these games we were doing much better doing computer chess games, than grasping or moving around and stuff like that. And what it tells you is that working with mental decision-making – we understand it better and, therefore, we can program it better. And if you have a lot of computer power, you can do a lot of searches and, therefore, you can do better than in this interaction with the physical world. That part is still not well understood. The flexibility that the biological systems have is phenomenal. And, of course, it’s a result of millions of years of evolution. But, nevertheless – and if you look at biology as I do in my spare time, it’s phenomenal that – how different creatures have developed their sensory and motor apparatus to adjust to their environment and survival. I give you just one example there is shrimp which has twelve different color filters in order to discriminate between their enemies and their friends through different color combinations. We have only three. And so you see this that the biological system adaptivity over long time has adapted to survival – to this environment. To me that is fascinating. That is fascinating and very inspirational.

Q:

And has that played into your work as well in various ways?

Ruzena Bajcsy:

Very much so. Very much so. Very much so. When I want to understand how many sensors you need to put on robots, what do you want to extract, it’s very clear to me that it’s very environment and task dependent. Yeah. It’s a basically an optimizational game that biology plays.

Q:

Would you call any of your projects, in particular, biologically inspired?

Ruzena Bajcsy:

Oh, yes. Oh, yes. Most of them. Now, I submit to you that frequently I had to reduce some of my activities to be pragmatic because of limitations of the hardware or the limitations of know-how, of the current scientific state of the art. I mean let’s face it, some of the tools – mathematical tools that we use today were not available 25 years ago, or so. And I’m not a mathematician. I wanted to be one, but I am not. So I just – I am a consumer of mathematics.

Q:

Could you give us some examples of particular projects that you did that were inspired by these biological –?

Ruzena Bajcsy:

One of them is this combining the tactile information with the visual information. I very quickly realized that the vision, which is indirect information, doesn’t give you enough information. The eyes can be fooled. So you need the contact in order to really measure the weight, for example. You can’t get weight from vision – the mass. So –

Q:

And so this is still things that you were doing at UPenn?

Ruzena Bajcsy:

Yes.

Q:

Okay. Mm-hmm.

Ruzena Bajcsy:

When I came here I changed, as one should, obviously. I started to focus on observing people. And so all my activities right now are centered on how we can use technology for improving communication between people. That’s why I started the tele-immersive laboratory. And I’m also putting sensors on people so that I can – for elderly, to monitor people. So that’s what I am now focusing. But I – people are very nice mechanical systems – I mean mechanical – dynamical systems. And it’s lot of fun to – and being an experimentalist, I basically build labs – facilities, so that we can get as best observations as we can. And then based on these observations, I use models – mathematical models that are out there, or that my students who know more math than I do, bring to bear to explain the data. So I am acting more as a experimental physicist, if you wish. In physics a result is not a result until you’re verified by some experiment, and the same is in chemistry. And I have that philosophy about my work, too, that result is not a result until I verify it with data. Sometimes I go the other way, I take observations and then I try to fit models that explain the data. But sometimes we have – I’m surrounded by all these theoreticians around me who have all kinds of models. And I say, “Well, how can I validate this model?” So that’s what I’m focusing since I came to Berkeley.

Berkeley Research and Applications

Q:

What was the first robot system that you built at Berkeley?

Ruzena Bajcsy:

At Berkeley?

Q:

Yeah.

Ruzena Bajcsy:

I didn’t build a robot system. No.

Q:

But the tele-immersive system –

Ruzena Bajcsy:

The tele-immersive laboratory, it’s 48 cameras surrounded – and the big thing that I struggled with – can I drink a little coffee? I mean even with the television. The big thing that I struggled in that lab was real time, capturing and reconstructing the stereonic [ph?] construction in the real time. So we started with four frames per second. Then we graduated to seven frames per second. Now I am happy to say we have twenty-five frames per second. So big, big accomplishment. And it wasn’t just by hard work. It was also clever thinking about how you use the raw data. And I was blessed by some very good students who helped, and did basically all the work. So I’m just a conductor.

Q:

Can you tell us the names of some of your students?

Ruzena Bajcsy:

Yes, with pleasure. So I started with Hong Seong who was a post-doc of mine. He came from Penn. And he built the first version of this. And then the real breakthrough we got when I started to work with Edgar Lobaton, L-O-B-A-T-O-N, and Ram Vasudevan who is still with me here. And they designed a new reconstruction algorithm that sped up the whole reconstruction. And let’s see. And then Gregorij Kurillo from Ljubljana came as a post-doc and he’s my senior engineer, who is basically keeping up the lab, improving, doing a lot of the applications. One thing is that these kind of facilities need money. So I always joke that basically the applications are selling, and then we do the science on the side.

Q:

What kind of applications do you work on there?

Ruzena Bajcsy:

Okay, so the first application we did with dancers, and we still continue because they were very receptive to the weaknesses of the technology. For example, in these four frames they had to move very slowly. But I also begun simultaneously collaboration with Professor Nahrstedt in Illinois, who happens to be my daughter, but –

Q:

I studied with her.

Ruzena Bajcsy:

Yeah, you did. I see, so you –

Q:

Multi-media.

Ruzena Bajcsy:

Okay, yes. So she built a similar lab – smaller, but similar lab. And so we had a joint grant, which enabled us to have a dancer in her lab and dancer in my lab and show that they can dance in one virtual space. So that was the first application. Then we moved from there on to – so we have currently application with archeologists who go out and they digitize all kinds of artifacts, caves and sculptures. And the collaborator is interested to connect with different archeologists so that they can be put into the same environment and then come up with a common interpretation because apparently they cannot agree on the common interpretation. So that’s one application. Now we have a collaboration with an aerospace company, EADS, which is international company. And they are interested to use this technology for discussing design distributively. And this technology is ideal for that. We have an application for medical application, physical therapy, okay, where remote coaches can guide people in this environment to – they can get into the same environment and tell them which way they should be moving, and so forth. And I’m hoping to have more of these applications, but right now – you know with all these applications, the biggest challenge is to find the right collaborators because I found that especially in the medical area, but any other area, as well, people are quite conservative. And they don’t have very much imagination what this can technology can do. So unless you show it to them, “Look you can come and do this, manipulate, move around, change this, change that, in the real time,” they just cannot comprehend. They cannot visualize what this can do.

So I’m spending a lot of time cultivating these collaborations, which is not trivial, but little by little. And the good news is that – as you can see, I have been doing this for at least five years now. And the good news is that the components are coming so you can buy now, commercially, a stereo camera, which will give you – and you can buy now 3D TVs are coming on the market. So the 3D – you know I have been accused often, why do you need 3D? Why not just have video conferencing. Okay? Well, if you have just video conferencing you cannot bring two people into the spatial configuration. You can have next to each other, but not in the same space. And that’s the difference. So I had to struggle with all this resistance for 3D. The other thing is where you need 3D is think about some of these games like basketball, or football, or soccer. And if you want to look at from any angle, once you have it in the computer, the 3D, then you can using just simple graphics rotate and look at the plays from any angle you wish to. So that’s another place.

But, of course, this technology is not cheap. And so that’s another hindrance before you can really make it big and widely available. Look what happened to robotics for all these years. I remember 25, 30 years ago we all were hoping that companies, industries would all be roboticized. Well it didn’t turn out that way. It turn out that the value that we were selling about robots, which was reconfigurability, reprogrammability, was just not there. And so it was cheaper for the automotive industry and all these big companies to make a fixed automation. Okay? Well now I think we are slowly getting there. But it’s still – with this tele-immersive technology we are learning that we still – it’s not a turnkey system where you can just plop it down, leave the room, and let people use it. Not just like a television, you just click. It’s not yet there, especially the cameras because it’s sensitive to the illumination, sensitive to the colors in your environment. So it ain’t that easy. So it will take couple of more years before we get there. So the components are coming, which is good news. And I am hoping that I can show the value of it in the health area. I am hoping, but you know the result – the proof to the pudding is to cook it and eat it.

Other Works and Collaborations

Q:

Who are some of the other people you’ve collaborated with over the years?

Ruzena Bajcsy:

Good question. Well, let’s see. Many, I have many collaborators. On this tele-immersion I really worked very closely and got very motivated by Henry Fuchs from North Carolina. We are not working now together, but I wish we could find someplace where we can work because he’s a very bright guy. And I really enjoy working with him. I still work with some of my colleagues at Penn, Kostas Daniilidis, Vijay Kumar, George Pappas. Here at Berkeley I collaborate with my colleague next door, Claire Tomlin, Shankar Sastry. Those are my closest collaborators. So who else I am collabor – Right now – Oh, yeah, yeah I collaborate with medical people. Okay, so I collaborated UC, Davis with Dr. Jay Han who is interested in monitoring these people who have neuromuscular diseases. And this technology is very good for that. And I’m hoping that we can develop a sort of ambulatory system that people can use at their home. They don’t have to come all the time to the clinic. And so – but, again, I tell you the interface is the – the human interface is really a challenge, how to make it so the mom or dad can use it easily with children or with elderly people. I’m actually focusing on these two categories, on children and elderly because they are the most vulnerable people. So that’s at UC, Davis. Here I’m collaborating with Dr. Edmund Seto who is a research professor at the School of Public Health. And I’m, of course, have been collaborating for years with Professor Lisa Wymore who is a professor of dance here at Berkeley. And, as I mentioned, I collaborated with Klara Nahrstedt, yeah. Who else do I collaborate? Well that’s about it. But the archeology is Professor Maurizio Forte from UC Merced.

Q:

And how do you find yourself with such cross-disciplinary collaborations? Do they find you? Do you find them? How do you, for example, start working on something like archeology?

Ruzena Bajcsy:

How did we start this archeology work? Okay, all right I know. Okay well, as you may know I came to Berkeley as a founding director of CITRIS. And the task of this institute was to see what information technology can do for societal needs. And so I had basically put to myself as a task – well one of the tasks was to cultivate with the relationship with companies because the original money was so that we can do more what they call useful research for industry. And we were measured by number of start-ups and that kind of thing. But I also felt that there was another community that served as a society, and that was the social science and humanities on the campus. That I felt didn’t take advantage of the information technology to full extent. So I started to look around on Berkeley campus and tried to see where information technology can play a role. And as I was digging around, that’s how I met Lisa Wymore. I give a presentation in the School of Humanities. Look this is what technology can do. And so – but as I was digging around I also found out that there was a group here led by this prof –

[recording ends abruptly]

Ruzena Bajcsy:

And one thing that you are in a position like director of CITRIS – I had a little money, so I could sort of give out money to seed some projects. So I connected with Lew Lancaster, who is an expert on Buddhism, and he was very much interested in taking these old maps of the Silk Road and digitizing them and show how the Silk Road evolved, because they were stationeries of these Buddhist monasteries. And I saw that was fascinating in application of information technology, and through him I started to think about more where information technology can play a role, and archaeology was a no-brainer. I mean, when you think about it, archaeologists destroy their place of inquiry as they are digging, so I said it makes sense to, as they are digging, take images and use our technology to reconstruct once they are done. So that’s how I connect with the – unfortunately, again, you need the right collaborators, so at Berkeley I approached the archaeologists and they said, “Just give us the money and leave us alone,” and I said, uh-uh, that’s not how I think of collaboration. So, in one of these meetings that we had, IT for humanities, I met this Professor Forte and then he took a job at Merced, so we started to work together.

Q:

How did you get involved with CITRIS to begin with?

Ruzena Bajcsy:

Oh, God. Well, before I came to Berkeley I was in Washington for three years in the National Science Foundation, and I was director of the computer science division, and concurrently Shankar Sastry was at DARPA heading the information technology director, and so we would meet in the – Washington has these interagency meetings, and we would meet there and we became friends. And then he left before I left NSF. He was called back to be a chair of the electrical engineering/computer science department, and at one point he said, “Well, Ruzena, they are looking for a director. Would you be interested?” and I said, “Sure, that sounds very interesting.” So I applied then. Then I was selected.

Q:

And what of their people are you working with within CITRIS?

Ruzena Bajcsy:

Well, of course as a former director, I am kind of involved indirectly. The current director is Paul Wright, so I talk to him from time to time. I don’t know directly. You see, in CITRIS, CITRIS is an organization that facilitates connections, so I don’t know how to answer your question. There is a person who is kind of a manager of these health-related projects, so I interact with him because of the UC Davis connection, but other than that, I mean, they are mostly – they organize meetings, and so there is no real – remember, I’m more interested in the substantive interaction, and these are administrators, so I don’t have very much to do with them.

Q:

And so, in a sense, after you finished being the director it did help you have some of these other collaborative relationships that you’re continuing?

Ruzena Bajcsy:

Yes.

Q:

So when you were at Penn, what were some of the other robotics projects you worked on?

Ruzena Bajcsy:

Oh, God, many. You have students so you have projects, right? Boy. Well, perhaps I should mention – well, I don’t know. It was always about this interaction between manipulation and vision. Then I had work with Jana Kosecka, who was my Ph.D. student who’s now professor at George Mason University. She worked on mobile. We had little mobile robots and vision and kind of – her job was to model the cooperation between two robots and when they move together like a flock of birds or somehow they coordinated what they have to tell each other and how independent they can be. It’s like the scouts meet me at place X and stuff like that, so that was – I don’t know what to tell you. I mean, there were multiple projects. I had a very outstanding student, actually perhaps the most successful academically student, Stéphane Mallat, who is a professor at L’Ecole Polytechnique in France, and his claim to fame is he introduced the wavelet transformation into the image processing. He had a very good mathematical background, so he was able to do some interesting stuff within computer vision, and we are still in contact and he’s going to come and visit me again and he’s doing some very interesting work again. So, that was Stéphane. Anyway, there are many, many, but one thing I should really mention that I had these two parallel lives in the medical image processing. I invented the anatomy atlas, the brain anatomy atlas, and that was with a Israeli student of mine, Haim Broit, B-R-O-I-T, who developed a kind of a deformable matching process that you take from – because one brain is similar to another brain but not exactly, so he modeled it as a membrane and developed the matching process to match, and that really picked up and now everybody’s using it in these MRI, CT, positron-emission tomography. This was in ’79, ’80s.

Future Research

Q:

Was there anything that you would’ve really liked to pursue in terms of research but didn’t get the chance to?

Ruzena Bajcsy:

I don’t think so. I will say that Penn was good for me for the – as you know, Penn engineering school, at least when I was there, was not very highly ranked, and so we did have a difficulty to get good students and all that. But I must say that Penn was good to me in the sense that they didn’t interfere with me, so they let me do whatever I chose to do. And as long as I was able to raise money and build lab and everything else, they didn’t interfere, and so in that sense they were very good, both colleagues as well as the dean, who was at that time Joe Bordogna, who was very supportive of me, and I am most grateful to him for that. And as the lab grew, the GRASP Lab, and the reputation grew, I was getting better and better students, as you can imagine, and I must say I’m very proud what the GRASP Lab has accomplished, and after Joe Bordogna left, I think this – well, was it Joe Bordogna or – no, there was another dean in between who didn’t do so much, but the current dean, Eduardo Glandt, has done really a very fine job in terms of raising money. They built new three buildings. They hired some very good people.

Now, of course, all the very good people were around the GRASP Lab, so you know how it is. I mean, good people attract good people, but the other thing that happened at Penn was that some of the old guard in the EE department retired, so they had the opportunity to hire and they really hired well. So, I will say that the Penn engineering school is on its way up, and that gives me a lot of pleasure because it really germinated from the GRASP Lab, which was from the very beginning a very interdisciplinary laboratory. I designed it that way, and so it benefited the whole school. We didn’t have chemical engineering, but we certainly had mechanical and electrical and computer people there from the very beginning, and as I said, now they are doing very well. And my husband complains that I left and so forth and so on, but I think I did the best thing for the place that I left, because I looked at other labs and if I would have stayed, it would have been all the time the Bajczy’s lab, and I didn’t want that happen. I really wanted the next generation to really feel ownership of the lab and do their own thing and put their mark, and I will tell you after I left Vijay Kumar took over, and he did a phenomenal job. I mean, he is fantastic, and he’s a good scientist and he’s a human on the human level, and so – and he hired George Pappas, and actually I hired Costas when I was still there, but they hired younger people. I don’t want to even name many of them, but they just hired outstanding faculty, and that’s what makes the place work.

So, Vijay did a phenomenal job, and after V.J. I think – was it George took over and now I think it’s Costas, but really the GRASP Lab is really basically helping Eduardo in the leadership. I mean, I think Vijay is the undergraduate dean and George is the graduate dean. I mean, what can I tell you? It’s all incestuous, but for good goals and purposes, so I’m very proud of – when you look back at my age and you say, well – people talk about legacy, right? I never think of legacy, but you look back and you look at your own children and you look back – I mean, these are my children and I’m very proud of them.

Q:

What are some of the challenges for you for the future?

Ruzena Bajcsy:

For the future? I don’t think you want to record that. My husband is one of the challenges. Well, I don’t know how many more years I have, but, anyway, I’m still in good health and my brains are working, so that’s okay. Here is what I would like to see happening with my little angle. First of all, I’d like to see this technology really be useful to people. I’m hoping that before I die that I will see this tele-immersive communication work, because in this globalized world if somehow we can bring people together more, then perhaps we can avoid all these wars and misunderstandings and all that.

To me communication is very important, and, yes, we have social networks and e-mail and all that, but especially when you talk about foreign cultures I am convinced that we communicate not only with verbally but also with our body, and that was my idea behind the tele-immersion, that you want to see and interact as a whole person, not just verbally. There is a lot of communication going on in terms of how we move, so – and all the modeling I am doing is in better understanding these movements. So, think about it. And this is really almost no practical application, but if I can model different cultural movements, I can have a quantitative comparison of cultures. Take Greek dances versus African dances versus Eastern European dances. There are certain messages there. If I can somehow mathematically capture that, perhaps that could contribute some better understanding. Look, I am an engineering scientist, so that’s all what I do, but if that can contribute I would be very gratified. So, this using technology for better communication – and that relates also to facilitate people to live alone more comfortably so that they don’t worry about – I’m a very independent person. I don’t want to live if I cannot be independent, and I think many people are like me, so if technology can help with that – and the pieces are there. The system is not yet there, but the pieces are out there. I think we will make a revolution in this regard. Now, the lastly I can tell you I’m right now writing this proposal. That’s what…

<crew talk>

Ruzena Bajcsy:

So, the proposal I’m writing right now is starting a collaboration with people from Oregon Health Center, and in a nutshell what excites me is that there is again psychological and behavioral evidence that your mental health is tightly connected to your physical health and vice-versa. So it’s a standard thing when you – in my age I have all these friends who are retired, and if they don’t move, they don’t exercise, they get mentally also depressed. So, what we hope to do is study this connection, and there are way how you can observe the mental alertness through various gains and so. And I also have technologies to study the physical. So if you can understand this formally this connection, then one can advise people, “Look, this is what’s happening in numbers.” Again, I am after quantitative understanding of this connection, which then can get feedback to people to say, “You don’t need to take pills. Just get off your butt and do something and you will feel much better here.” And why you can ask, why this hasn’t been done? I will tell you why: because you have two different communities. The cognitive part is studied by AI, the artificial intelligence, and the physical part is studied by the control people and they hardly ever talk to each other. They use different mathematical tools and there are two different communities, so I’d somehow like to bridge it. So that’s what I would like to see before I kick the bucket.

Reflection

Q:

And in terms of just robotics in general, what do you think are some of the – where should it be going or where…?

Ruzena Bajcsy:

Oh, boy. That’s a loaded question.

Q:

Or, where would you like to see it going, not where it should. It’s kind of…

Ruzena Bajcsy:

Well, there are many different levels. So you have robotics. I’m really concerned about the young people in Oakland who have no jobs, and I’m not sure robotics will help them, because these people – for whatever reason they are not able to go beyond maybe high school, whether – it’s irrelevant why, but they are not. So, one thing that I have been thinking but I don’t have the means or energy to do that, that it would be nice if we could use this technology for training them in trades, how to be a manual electrician or how to be a mason or how to be a whatever trade with your hands. And, again, because I work with this three-dimensional world, this technology should be able to help them that even in this virtual environment they could try to play with some of these physical objects, but I’m not sure. Again, it’s a question of money and so forth.

So, in manufacturing I think robotics is going to gain credit namely. People are making efforts making these robots more reconfigurable and easy to program and easy to use, and I think that’s coming, but I think more interestingly the automobile industry is getting much more roboticized, and I’m involved in how to observe people when they are distracted with their cell phones or with whatever else and give them a feedback, either a sound or something, don’t do this, or, in fact, slowing you down and stuff like that. So, the automobile industry is starting to use a lot of our technologies, either alarming you, “You are too close to somebody!” or parallel parking and stuff like that, and that’s already the industry is starting to use it. So, I see a lot of applications there, also monitoring traffic, so that’s all in transportation. In domestic robots it’s not clear to me, and I tell you why. One is safety. It has to be simple. That’s why this iRobot made it, because it’s $200.00 or $150.00 or so, and it’s just very dumb, just doing vacuum cleaning. People talk about toilet cleaners and kitchen and all that. The variety of environments is so huge that it’s not obvious to me how soon that is going to be happening. On the other hand, I already see some reasonable applications in these institutions like hospitals delivering meals and stuff like that. So, airlines probably will be using more robotics. It’s all a function of easy to use, robustness and simplicity. And 30 years ago when I started, 40 years ago, we were dreaming about these RUR robots that will do everything and anything. I don’t think that’s going to happen. I think we will have specialized robots or robotic technologies rather than a universal robot that will do…

Q:

Where did you first encounter the RUR, Rossum’s Universal Robots?

Ruzena Bajcsy:

Oh, boy.

Q:

That was a Czech play.

Ruzena Bajcsy:

I remember it was…

Q:

Čapek.

Ruzena Bajcsy:

…Karel Čapek, and that was just before the Second World War, and my parents had his books, so I think it was in the ’40s or so.

Q:

Do you think that influenced your interest in robots?

Ruzena Bajcsy:

I am not sure. I am not sure. I cannot remember that far.

Q:

That was maybe your first kind of being aware of something called a robot or no?

Ruzena Bajcsy:

Yeah. I mean, it was a book that kind of my parents said “read it.” He had another book about dogs. I remember pictures of dogs. He loved dogs, or maybe that was his brother. There were two brothers, Čapek brothers, and the tragedy – luckily one of them died before Hitler came to Czechoslovakia, because they were very anti-Nazis, and the other one, I think he died in a concentration camp. Bad time.

Q:

Well, I think we’ve exhausted the various things that we planned to ask.