Files have been the common element of computing since at least the 1960s. Files existed before disk drives and file systems, as one could put multiple files on a magnetic tape.
MS-DOS used files. Windows used files. OS/2 used files. (Even the p-System used files.)
Files were the unit of data storage. Applications read data from files and wrote data to files. Applications shared data through files. Word processor? Files. Spreadsheet? Files. Editor? Files. Compiler? Files.
The development of databases saw another channel for sharing data. Databases were (and still are) used in specialized applications. Relational databases are good for consistently structured data, and provide transactions to update multiple tables at once. Microsoft hosts its Team Foundation on top of its SQL Server. (Git, in contrast, uses files exclusively.)
Despite the advantages of databases, the main method for storing and sharing data remains files.
Until now. Or in a little while.
Cloud computing and web services are changing the picture. Web services are replacing files. Web services can store data and retrieve data, just as files. But web services are cloud residents; files are for local computing. Using URLs, one can think of a web service as a file with a rather funny name.
Web services are also dynamic. A file is a static collection of bytes: what you read is exactly was was written. A web service can provide a set of bytes that is constructed "on the fly".
Applications that use local computing -- desktop applications -- will continue to use files. Cloud applications will use web services.
Those web services will be, at some point, reading and writing files, or database entries, which will eventually be stored in files. Files will continue to exist, as the basement of data storage -- around, but visited by only a few people who have business there.
At the application layer, cloud applications and mobile applications will use web services. The web service will be the dominant method of storing, retrieving, and sharing data. It will become the dominant method because the cloud will become the dominant location for storing data. Local computing, long the leading form, will fall to the cloud.
The default location for data will be the cloud; new applications will store data in the cloud; everyone will think of the cloud. Local storage and local computing will be the oddball configuration. Legacy systems will use local storage; modern systems will use the cloud.
Showing posts with label web services. Show all posts
Showing posts with label web services. Show all posts
Monday, September 25, 2017
Sunday, April 17, 2016
After the spreadsheet
The spreadsheet is a wonderful thing. It performs several functions: it holds and organizes data, it specifies calculations, and it presents results. All of these functions, wrapped into a single package, provides convenience to users. Yet that convenience of the single package will be the spreadsheet's undoing.
For the individual, the spreadsheet is a useful tool. But for the enterprise, the spreadsheet creates perhaps more problems than it solves. Since a spreadsheet file contains the data, formulas, and presentation of data, they are often replicated to share with co-workers (usually via e-mail) and duplicated to process different sets of data (the spring sales figures and then the summer sales figures, for example).
The replication of spreadsheets via e-mail can me mitigated by the use of shared file locations ("network drives") and by online versions of spreadsheets which allow for multiple users. But the bigger problem is the duplication of spreadsheets with minor changes.
The duplication of spreadsheet means the duplication of not only the data (which is often changed) but also the duplication of the formulas and the presentation (which often do not change). Since a spreadsheet contains all three components, a new version of data requires a new copy of all components. There is no way to share only one component, no way to share formulas against new data, or different presentations against data and formulas. This means that, over time, an enterprise of any size accumulates multiple spreadsheets with different data and duplicate formulas and macros -- at least you hope that they are duplicate copies.
The design of spreadsheets -- containing the data, formulas, and presentation in one package -- is a holdover from the days of Visicalc and Lotus 1-2-3. Those programs were developed for the Apple II and the IBM PC. With their ability to run only one program at a time, putting everything into one program made sense -- using one program for data, another for calculation, and a third for presentation was awkward and time-consuming. But that applies to the old single-tasking operating systems. Windows and Mac OS and Linux allow for multiple programs to run at the same time, and windowing systems allow for multiple programs to present information to the user at the same time.
If spreadsheets were being invented now in the age of web services and cloud systems and multi-window displays, their design would probably be quite different. Instead of a single program that performed all functions and a single file that contained data, formulas and presentation, we might have something very different. We might create a system of web services, providing data with some and performing calculations with others. The results could be displayed by yet other functions in other windows, possibly published for co-workers to view.
Such a multi-component system would follow the tenets of Unix, which recommends small, independent programs that read data, perform some processing, and provide data. The data and computations could be available via web services. A central service could "fan out" requests to collect data from one or more services, send that data through one or more computing services, and the provide the data to a presentation mechanism such as a graph in a window or even a printed report.
By separating the formulas and macros from the data, we can avoid needless duplication of both. (While most cases see the duplication of formulas to handle different data sets, sometimes different formulas can be applied to the same data.)
Providing data via web services is easy -- web services do that today. There are even web services to convert data into graphs. What about calculations? What language can be used to perform computations on data sets?
The traditional languages of C# and Java are not the best here; we're replacing spreadsheets with something equally usable by non-programmers (or at least similarly usable). The best candidate may be R, the statistical-oriented language. R is established, cross-platform, and capable. It's also a high-level language, close the the formulas of spreadsheets (and more powerful that Microsoft's VBA, which is used for macros in Excel).
Replacing spreadsheets with a trio of data management, computation, and presentation tools will not be easy. The advantages of the spreadsheet include convenience and familiarity. The advantages of separate components are better integration in cloud systems, leveraging of web services, and easier audits of formulas. It may not happen soon, but I think it will happen eventually.
For the individual, the spreadsheet is a useful tool. But for the enterprise, the spreadsheet creates perhaps more problems than it solves. Since a spreadsheet file contains the data, formulas, and presentation of data, they are often replicated to share with co-workers (usually via e-mail) and duplicated to process different sets of data (the spring sales figures and then the summer sales figures, for example).
The replication of spreadsheets via e-mail can me mitigated by the use of shared file locations ("network drives") and by online versions of spreadsheets which allow for multiple users. But the bigger problem is the duplication of spreadsheets with minor changes.
The duplication of spreadsheet means the duplication of not only the data (which is often changed) but also the duplication of the formulas and the presentation (which often do not change). Since a spreadsheet contains all three components, a new version of data requires a new copy of all components. There is no way to share only one component, no way to share formulas against new data, or different presentations against data and formulas. This means that, over time, an enterprise of any size accumulates multiple spreadsheets with different data and duplicate formulas and macros -- at least you hope that they are duplicate copies.
The design of spreadsheets -- containing the data, formulas, and presentation in one package -- is a holdover from the days of Visicalc and Lotus 1-2-3. Those programs were developed for the Apple II and the IBM PC. With their ability to run only one program at a time, putting everything into one program made sense -- using one program for data, another for calculation, and a third for presentation was awkward and time-consuming. But that applies to the old single-tasking operating systems. Windows and Mac OS and Linux allow for multiple programs to run at the same time, and windowing systems allow for multiple programs to present information to the user at the same time.
If spreadsheets were being invented now in the age of web services and cloud systems and multi-window displays, their design would probably be quite different. Instead of a single program that performed all functions and a single file that contained data, formulas and presentation, we might have something very different. We might create a system of web services, providing data with some and performing calculations with others. The results could be displayed by yet other functions in other windows, possibly published for co-workers to view.
Such a multi-component system would follow the tenets of Unix, which recommends small, independent programs that read data, perform some processing, and provide data. The data and computations could be available via web services. A central service could "fan out" requests to collect data from one or more services, send that data through one or more computing services, and the provide the data to a presentation mechanism such as a graph in a window or even a printed report.
By separating the formulas and macros from the data, we can avoid needless duplication of both. (While most cases see the duplication of formulas to handle different data sets, sometimes different formulas can be applied to the same data.)
Providing data via web services is easy -- web services do that today. There are even web services to convert data into graphs. What about calculations? What language can be used to perform computations on data sets?
The traditional languages of C# and Java are not the best here; we're replacing spreadsheets with something equally usable by non-programmers (or at least similarly usable). The best candidate may be R, the statistical-oriented language. R is established, cross-platform, and capable. It's also a high-level language, close the the formulas of spreadsheets (and more powerful that Microsoft's VBA, which is used for macros in Excel).
Replacing spreadsheets with a trio of data management, computation, and presentation tools will not be easy. The advantages of the spreadsheet include convenience and familiarity. The advantages of separate components are better integration in cloud systems, leveraging of web services, and easier audits of formulas. It may not happen soon, but I think it will happen eventually.
Tuesday, June 30, 2015
Apple's browser
Does Apple need a browser?
Apple has Safari, but maintains it poorly. Is Apple serious about its browser?
Come to think of it, why does Apple need a browser?
I can think of several reasons:
Ego Apple has to have a browser to satisfy its ego. It needs to have a browser to meet some internal need.
Freudian? Yes. Reasonable? No.
Apple is a smart company. It doesn't invest in products to suit its ego. It invests to improve revenue.
Keeping up with Microsoft Microsoft has a browser. Apple wants, at some level, to compete with Microsoft. Therefore, Apple needs a browser.
Doubtful. Apple doesn't have to match Microsoft one-for-one on features. They never have.
Superlative web experience Apple knows Mac OSX and iOS better then anyone. They, and only they, can build the browser that provides the proper user experience.
Possible. But only if Apple thinks that a good web experience is necessary.
Avoid dependence on others Max OSX (and iOS), despite Apple's most fervent wishes, still needs a browser. Without an Apple browser, Apple would have to rely on another browser. Perhaps Google Chrome. Perhaps Mozilla Firefox. But relying on Google is risky -- Google and Apple are not the best of friends. Relying on Mozilla is also risky, but in another sense: Mozilla may not be around much longer, thanks to the popularity of other browsers (of which Safari is one).
All of these strategies have one thing in common: the assumption that Apple considers the web important.
I'm not sure Apple thinks that. It may be that Apple thinks, in the long run, that the web is unimportant. Apple focusses on native apps, not HTML 5 apps. The dominant app design processes data on the local device and uses the cloud for storage, but nothing more. Apple doesn't provide cloud services for computing. Apple has no service that matches Windows Azure or Google's computer engine. In Apple's world, devices compute and the cloud stores.
* * * * *
The more I think about it, the more I convince myself that Apple doesn't need a browser. Apple has already delayed several improvements to Safari. Maybe Apple thinks that Safari is good enough for the rest of us.
In the Apple ecosystem, they may be right.
Apple has Safari, but maintains it poorly. Is Apple serious about its browser?
Come to think of it, why does Apple need a browser?
I can think of several reasons:
Ego Apple has to have a browser to satisfy its ego. It needs to have a browser to meet some internal need.
Freudian? Yes. Reasonable? No.
Apple is a smart company. It doesn't invest in products to suit its ego. It invests to improve revenue.
Keeping up with Microsoft Microsoft has a browser. Apple wants, at some level, to compete with Microsoft. Therefore, Apple needs a browser.
Doubtful. Apple doesn't have to match Microsoft one-for-one on features. They never have.
Superlative web experience Apple knows Mac OSX and iOS better then anyone. They, and only they, can build the browser that provides the proper user experience.
Possible. But only if Apple thinks that a good web experience is necessary.
Avoid dependence on others Max OSX (and iOS), despite Apple's most fervent wishes, still needs a browser. Without an Apple browser, Apple would have to rely on another browser. Perhaps Google Chrome. Perhaps Mozilla Firefox. But relying on Google is risky -- Google and Apple are not the best of friends. Relying on Mozilla is also risky, but in another sense: Mozilla may not be around much longer, thanks to the popularity of other browsers (of which Safari is one).
All of these strategies have one thing in common: the assumption that Apple considers the web important.
I'm not sure Apple thinks that. It may be that Apple thinks, in the long run, that the web is unimportant. Apple focusses on native apps, not HTML 5 apps. The dominant app design processes data on the local device and uses the cloud for storage, but nothing more. Apple doesn't provide cloud services for computing. Apple has no service that matches Windows Azure or Google's computer engine. In Apple's world, devices compute and the cloud stores.
* * * * *
The more I think about it, the more I convince myself that Apple doesn't need a browser. Apple has already delayed several improvements to Safari. Maybe Apple thinks that Safari is good enough for the rest of us.
In the Apple ecosystem, they may be right.
Sunday, June 14, 2015
Data services are more flexible than files
Data services provide data. So do files. But the two are very different.
In the classic PC world ("classic" meaning desktop applications), the primary storage mechanism is the file. A file is, at its core, a bunch of bytes. Not just a random collection of bytes, but a meaningful collection. That collection could be a text file, a document, a spreadsheet, or any one of a number of possibilities.
In the cloud world, the primary storage mechanism is the data service. That could be an SQL database, a NoSQL database, or a web service (a data service). A data service provides a collection of values, not a collection of bytes.
Data services are active things. They can perform operations. A data service is much like a query in an SQL database. (One may think of SQL as a data service, if one likes.) You can specify a subset of the data (either columns or rows, or both), the sequence in which the data appears (again, either columns or rows, or both), and the format of the data. For sophisticated services, you can collect data from multiple sources.
Data services are much more flexible and powerful than files.
But that's not what is interesting about data services.
What is interesting about data services is the mindset of the programmer.
When a programmer is working with data files, he must think about what he needs, what is in the file, and how to extract what he needs from the file. The file may have extra data (unwanted data rows, or perhaps undesired headings and footings). The file may have extra columns of data. The data may be in a sequence different from the desired sequence. The data may be in a format that is different from what is needed.
The programmer must compensate for all of these things, and write code to handle the unwanted data or the improper formats. Working with files means writing code to match the file.
In contrast, data services -- well-designed data services -- can format the data, filter the data, and clean the data for the programmer. Data services have capabilities that files do not; they are active and can perform operations.
A programmer using files must think "what does the file provide, and how can I convert it to what I need?"; a programmer using data services thinks "what do I need?".
With data services, the programmer can think less about what is available and think more about what has to be done with the data. If you're a programmer or a manager, you understand how this change makes programmers more efficient.
If you're writing code or managing projects, think about data services. Even outside of the cloud, data services can reduce the programming effort.
In the classic PC world ("classic" meaning desktop applications), the primary storage mechanism is the file. A file is, at its core, a bunch of bytes. Not just a random collection of bytes, but a meaningful collection. That collection could be a text file, a document, a spreadsheet, or any one of a number of possibilities.
In the cloud world, the primary storage mechanism is the data service. That could be an SQL database, a NoSQL database, or a web service (a data service). A data service provides a collection of values, not a collection of bytes.
Data services are active things. They can perform operations. A data service is much like a query in an SQL database. (One may think of SQL as a data service, if one likes.) You can specify a subset of the data (either columns or rows, or both), the sequence in which the data appears (again, either columns or rows, or both), and the format of the data. For sophisticated services, you can collect data from multiple sources.
Data services are much more flexible and powerful than files.
But that's not what is interesting about data services.
What is interesting about data services is the mindset of the programmer.
When a programmer is working with data files, he must think about what he needs, what is in the file, and how to extract what he needs from the file. The file may have extra data (unwanted data rows, or perhaps undesired headings and footings). The file may have extra columns of data. The data may be in a sequence different from the desired sequence. The data may be in a format that is different from what is needed.
The programmer must compensate for all of these things, and write code to handle the unwanted data or the improper formats. Working with files means writing code to match the file.
In contrast, data services -- well-designed data services -- can format the data, filter the data, and clean the data for the programmer. Data services have capabilities that files do not; they are active and can perform operations.
A programmer using files must think "what does the file provide, and how can I convert it to what I need?"; a programmer using data services thinks "what do I need?".
With data services, the programmer can think less about what is available and think more about what has to be done with the data. If you're a programmer or a manager, you understand how this change makes programmers more efficient.
If you're writing code or managing projects, think about data services. Even outside of the cloud, data services can reduce the programming effort.
Thursday, April 30, 2015
Files are static, requests are dynamic
The transition from desktop or web applications to mobile/cloud systems is more than the re-organization of programs. It is a change from data sources: desktop and web applications often store data in files, and mobile/cloud systems store data via web services.
Files are static things. They perform no actions by themselves. A program can read the contents of a file and take action on those contents, but it must consume the contents as they exist. The file may contain just the data that a program needs, or it may contain more, or less. For example, a file containing a Microsoft Word document actually contains the text of the document, revisions to the text, information about fonts and formatting, and meta-information about the author.
A program reading the contents of that file must read all of that information; it has no choice. If the task is to extract the text -- and only the text -- the program must read the entire file, revisions and fonts and meta-information included. If we want only the meta-information, the program must read the entire file, text and revisions... you get the idea.
(The more recent DOCX format does isolate the different sets of information, and makes reading a subset of the file easier. The older DOC format required reading and interpreting the entire file to obtain any part of the file.)
Web services, used by mobile/cloud systems, are not static but dynamic. (At least they have the possibility of being dynamic. You can build web services that mimic files, but you probably want the dynamic versions.)
A web service can be dynamic because there is another program processing the request and creating the response. A web service to read a document can do more than simply return the bytes in the document file. It can perform some processing on your behalf. It can accept instructions, such as "give me the text" or "only the meta-information, please". It can do these things on our behalf. We can delegate them to the web service, and our job becomes easier.
Astute readers will observe that my arrangement of dynamic web services does not reduce the work involved, it merely shifts work to different parts of the system. (The web service must still read the entire document, pick out the bits of interest, and send those to us.) That is true. Yet it is also true that once in place, the web service provide an interface to reading (and writing) documents, and we may then choose to change the implementation of that storage.
With document web services in place, our applications are completely ignorant of the storage format and the web services may change that format to suit their needs. A new version of web services may store documents not in files but in databases, or JSON files, or any method appropriate. I'm pretty sure that Google Docs uses this approach, and I suspect Microsoft's Office 365, if not using it now, will use it soon.
Moving from desktop and web to mobile/cloud lets us do many things. It lets us do many things that we do today but differently. Look at the possibilities, and look at the savings in effort and cost.
Files are static things. They perform no actions by themselves. A program can read the contents of a file and take action on those contents, but it must consume the contents as they exist. The file may contain just the data that a program needs, or it may contain more, or less. For example, a file containing a Microsoft Word document actually contains the text of the document, revisions to the text, information about fonts and formatting, and meta-information about the author.
A program reading the contents of that file must read all of that information; it has no choice. If the task is to extract the text -- and only the text -- the program must read the entire file, revisions and fonts and meta-information included. If we want only the meta-information, the program must read the entire file, text and revisions... you get the idea.
(The more recent DOCX format does isolate the different sets of information, and makes reading a subset of the file easier. The older DOC format required reading and interpreting the entire file to obtain any part of the file.)
Web services, used by mobile/cloud systems, are not static but dynamic. (At least they have the possibility of being dynamic. You can build web services that mimic files, but you probably want the dynamic versions.)
A web service can be dynamic because there is another program processing the request and creating the response. A web service to read a document can do more than simply return the bytes in the document file. It can perform some processing on your behalf. It can accept instructions, such as "give me the text" or "only the meta-information, please". It can do these things on our behalf. We can delegate them to the web service, and our job becomes easier.
Astute readers will observe that my arrangement of dynamic web services does not reduce the work involved, it merely shifts work to different parts of the system. (The web service must still read the entire document, pick out the bits of interest, and send those to us.) That is true. Yet it is also true that once in place, the web service provide an interface to reading (and writing) documents, and we may then choose to change the implementation of that storage.
With document web services in place, our applications are completely ignorant of the storage format and the web services may change that format to suit their needs. A new version of web services may store documents not in files but in databases, or JSON files, or any method appropriate. I'm pretty sure that Google Docs uses this approach, and I suspect Microsoft's Office 365, if not using it now, will use it soon.
Moving from desktop and web to mobile/cloud lets us do many things. It lets us do many things that we do today but differently. Look at the possibilities, and look at the savings in effort and cost.
Monday, April 6, 2015
Web services use the Unix way
Web services use a very different mindset than the typical Windows application. Windows applications are large, all-encompassing systems that contain everything they need.
Web services, in contrast, are small fractions of a system. A complete system can be composed of web services, but a web service is not a complete system.
We've seen this pattern before. Not in Windows, nor in PC-DOS or MS-DOS, but in Unix (and now Linux).
"The Art of Unix Programming" by E.S. Raymond covers the topic well. The author describes the philosophy of small, connectable programs. Each program does one thing well, and systems are built from these programs. Anyone who has worked with the Unix (or Linux) command line knows how to connect multiple programs into a larger system.
We don't need to discover the properties of good web services. We don't need to re-invent the techniques for desiging good web services. The properties and techniques already exist, in the command-line philosophy of Unix.
Tuesday, October 11, 2011
SOA is not DOA
SOA (service oriented architecture) is not dead. It is alive and well.
Mobile apps use it. iPhone apps that get data from a server (e-mail or Twitter, for example) use web services -- a service oriented architecture.
SOA was the big thing back in 2006. So why do we not hear about it today?
I suspect it had nothing to do with SOA's marketability.
I suspect that no one talks about SOA because no one makes money from it.
Object oriented programming was an opportunity to make money. Programmers had to learn new techniques and new languages; tool vendors had to provide new compilers, debuggers, and IDEs.
Java was a new programming language. Programmers had to learn it. Vendors provided new compilers and IDEs.
UML was big, for a while. Vendors provided tools; architects, programmers, and analysts learned it.
The "retraining load" for SOA is smaller, limited mostly to the architects of systems. (And there are far fewer architects than programmers or analysts.) SOA has no direct affect on programmers.
With no large-scale training programs for SOA (and no large-scale training budgets for SOA), vendors had no incentive to advertise it. They were better off hawking new versions of compilers.
Thus, SOA quietly faded into the background.
But it's not dead.
Mobile apps use SOA to get work done. iPhones and Android phones talk to servers, using web services. This design is SOA. We may not call it that, but that's what it is.
When the hype of SOA vanished, lots of companies dropped interest in SOA. Now, to move their applications to the mobile world, they will have to learn SOA.
So don't count SOA among the dead.
On the other hand, don't count on it for your profits. You need it, but it is infrastructure, like electricity and running water. I know of few companies that count on those utilities as a competitive advantage.
Mobile apps use it. iPhone apps that get data from a server (e-mail or Twitter, for example) use web services -- a service oriented architecture.
SOA was the big thing back in 2006. So why do we not hear about it today?
I suspect it had nothing to do with SOA's marketability.
I suspect that no one talks about SOA because no one makes money from it.
Object oriented programming was an opportunity to make money. Programmers had to learn new techniques and new languages; tool vendors had to provide new compilers, debuggers, and IDEs.
Java was a new programming language. Programmers had to learn it. Vendors provided new compilers and IDEs.
UML was big, for a while. Vendors provided tools; architects, programmers, and analysts learned it.
The "retraining load" for SOA is smaller, limited mostly to the architects of systems. (And there are far fewer architects than programmers or analysts.) SOA has no direct affect on programmers.
With no large-scale training programs for SOA (and no large-scale training budgets for SOA), vendors had no incentive to advertise it. They were better off hawking new versions of compilers.
Thus, SOA quietly faded into the background.
But it's not dead.
Mobile apps use SOA to get work done. iPhones and Android phones talk to servers, using web services. This design is SOA. We may not call it that, but that's what it is.
When the hype of SOA vanished, lots of companies dropped interest in SOA. Now, to move their applications to the mobile world, they will have to learn SOA.
So don't count SOA among the dead.
On the other hand, don't count on it for your profits. You need it, but it is infrastructure, like electricity and running water. I know of few companies that count on those utilities as a competitive advantage.
Subscribe to:
Posts (Atom)