Guest Article: Storing CAD Databases in the Cloud

Guest Article: Storing CAD Databases in the Cloud

  • Comments 9
  • Likes

Author: Deelip Menezes

The following article is authored by PLM industry analyst and CAD expert Deelip Menezes. His complete bio can be found here

 

For some time now, Computer-Aided Design (CAD) systems have been storing project data in individual files. But recent developments in product lifecycle management platforms, such as Dassault Systemes CATIA V6, have changed the game for storing your CAD data. Dassault Systemes CATIA V6 stores all project data in database, and no longer uses a native file format for their MCAD system storage. So, to be clear we’re not talking about dumping data into a single field as a BLOB (binary large object); we’re talking about a true database implementation for improved management, performance, and scalability. This means that CATIA V6 stores geometric and product data in individual records within database, and that’s a good thing.

All this takes me back to 1997 when I was working as a computer programmer for a company specializing in commuter bus bodies, where we had tried something very similar. Basically, as a result of multiple iterations of a design process we found we had eventually built an extensive design system, using AutoCAD Release 12 and FoxPro 2.6 for DOS, that could automatically create the geometry for practically all of our bus parts and assembly units. The process began with a Designer entering numerical data into a FoxPro form (Think: basic parameters of the bus such as wheelbase, number and types of seats, etc…), so the data was stored in a grouping of FoxPro DBF files. Then, the Designer using AutoCAD would run a set of AutoLISP programs created by our Developers. These programs would query the data in the DBF files and eventually produce actual drawings (DWG files) of the parts and system assemblies to be fabricated—resulting in output of the basic business logic that we had intended. From the Designer’s perspective, the creation of DWG files was of nearly transparent because they could be created at any time using a combination of data in the DBF files and our in-house AutoLISP programs. These programs eventually evolved to a point where we were able to place geometric data back into the database. And, over time the Designers could revise of drawings in FoxPro and never touch AutoCAD. This framework also allowed them to extensively reuse data. They could take pieces of data from various projects and mix them together to come up with a new project thereby saving time. Many times the design of an entire bus was done entirely in FoxPro. AutoCAD was used like a printer to simply churn out DWG files.

But here is the best part: anyone who was not working in the design department, and had absolutely no CAD experience, could query and even edit the geometric and product data stored in the database. This would ultimately result in actual changes in the geometry of the parts and assemblies. These “non-Designers” were absolutely amazed at the way they could affect the design of the various parts and assemblies of a bus with little-to-no knowledge of AutoCAD. As you can imagine, we had to put in place various checks and balances in place so that they didn’t mess things up, but overall our results were positive. I suppose my point is this: the actual geometric data could be entered and modified independently simply because it was stored in a database. Had we stayed with the system of single DWG files as system of data entry, data independence would not have been possible. Using a database was the key.

This was more than a decade ago using software and technologies that are long gone, and nearly forgotten. Imagine what we could have done using the power packed and highly scalable database technologies of today. Held to the latest standards, our system seems crude, and furthermore limited our collaboration to within the company Intranet. So, let’s take it a step further, and imagine what could be done with cloud based databases like Microsoft SQL Azure. Imagine the CAD and product-specific data of a company’s products stored in a database in the Cloud and managed by a Cloud-based PLM system. Imagine this data accessible all over the world instantly by anyone with the appropriate access. This is true collaboration—data utilization generating real results.

When you think about it, this completely overhauls the concept of search geometry. There is no longer a need to load each individual CAD file into memory, parse, and then look for parameters and features that match the search criteria. And, consider the impact to computationally expensive feature recognition. All the geometric data to be searched is already available and indexed in the database. For example, if I wanted to find every part within in my company that has four or more holes of a specific diameter (…let’s say 15mm), I wouldn’t have to open the thousands of part files and run a query on each and every one of them. No, I would simply run a SQL query and receive instantaneous results.

There’s one other thing, and this comes as no surprise, but the simple management of CAD data is better when it resides in a database. Just like you can lock certain records in a database, or set up rules determining which records can be changed and by whom, you can also lock parts of your model. Instead of checking files in and out, you can check the features of a model in and out. In other words, revisions need not be tied to an single file. You can have revisions for different features of your model. This level of granularity opens up possibilities that simply do not exist today.

This also opens up CAD data to non-CAD users. Like I mentioned in my FoxPro example above, geometry could be edited without ever touching the CAD system that created it. Now, this kind of automation can be done quite simply and the permutations are limitless. Back in 1997, records in a FoxPro DBF file determined what the geometry in a DWG file would look like. I remember writing a nesting program to optimally cut the floor plates that were to be riveted to the floor structure of the bus. The program would place and orient the plates on blank sheets so that the scrap was minimized. I wrote that program in FoxPro, not AutoLISP, because after the floor plates were cut the remnant sheets were to be recorded back into the database for future use. The user of that program was actually creating geometry without even knowing it. The records that he was adding to the DBF files represented rectangles and polylines and he had never seen AutoCAD in his life. Oftentimes, the user was a data entry operator sitting on a computer far away from the design department. Quite frankly, I don’t think he or she knew what they was doing—no offense intended—and it really didn’t matter because my program prevented them from doing something stupid.

Okay, so CAD data stored in a database is fundamental step in the right direction, but what if the database was located on the cloud? Earlier I had mentioned SQL Azure and how it could be used to drive a cloud based PLM system. Let’s take a deep look. Microsoft SQL Azure Database is a cloud-based database built on SQL Server technologies. It is offered as a Platform as a Service (PaaS ). This means that it does not require any physical administration: the software is installed, updated, and managed by Microsoft. However, if you want to manage the database yourself, Microsoft has something that it has codenamed “Project Houston” which is basically a lightweight and simple database management tool for SQL Azure databases.

One of the benefits of cloud computing is that it gives the sense of infinite computing resources because of its ability to provision the resources when needed and to scale. The same holds true for SQL Azure, databases can be scaled up or down according to your demand. Back in 1997, we continuously found ourselves battling the limitations of FoxPro and DOS, and this was frustrating and eventually hindered our ability to innovate at the rate we needed. But recent database implementations, like Microsoft SQL Azure, are far more scalable and flexible, and this bodes well for the future of innovation. Imagine all geometric and product data of an Airbus A380 stored in a database like SQL Azure on the Cloud. This reminds me, the A380 was significantly delayed due to problems linked to the 330 miles of electrical wiring in each aircraft (for those counting, that turns out to be about 100,000 wires and 40,300 connectors). In the end, the German and Spanish Airbus facilities continued to use the older CATIA V4, while the British and French sites had moved to CATIA V5. In actuality, the mix of CATIA versions resulted in poor data exchange between regional sites.

Fast-forward to today and imagine if there were no regional data exchange limitations  and all data was stored in a database in the Cloud. The result? Everyone working on the same data located in the same source. Of course, there would be the usual revision control, but my point it there would be no need for data migration and all the potential million (perhaps billion) dollar mistakes that could come with it. The Airbus situation led to large scale cancellations of orders and a 26% drop in the share price of Airbus’ parent EADS. That led to the departure of the CEO’s of EADS and Airbus as well as the A380 program manager. Of course there were other factors that contributed to these happenings. But Airbus did point to the problem data exchange between CATIA V4 and V5 as one of the reasons for their delays.

It’s clear that data stored in a file has limitations. We are at a time where most agree that “If it’s data, then it belongs in a database”. And soon it will be said that “if data resides on the cloud, then it should in a database in the Cloud.”

 

Comments
  • OK, now I am a cad user and this list of questions yet to be answered is primarily related to cad on the cloud but much of it is appropriate to any serious data transfers over the cloud. No company seems to want to talk about the large number of pitfalls which are glossed over with an agressive ad campaign by those who wish to get perpetual income where users have no choice but to send in regular payments instead of being able to buy permanat seats of whatever. So here is a list of relevant questions for you cloud proponents.

      Is there a guaranteed minimum through put for data exchange irregardless of my ISP so I can benefit from the promised increased capabilities.

    If a user wishes to use a different service pack of the software for any reason does he have a choice or is it single version only dependent upon the cloud companies selection.

    Is the user guaranteed security backed with compensation for any resulting damages from breaches due to having to go online.

    Is the user guaranteed compensation for downtime due to web issues.

    How can the cloud possibly do anything for me as a user when cad is still basically a one core program and multicores don't matter much compared to basic single core processor speed.

    Does the cloud offer multicore capabilities for cad you are unwilling to sell me with stand alone seats.

    Will rendering which is one of the few areas suitable to multicore environments be guaranteed to be faster than what I can do with my workstation considering ISP problems and data throughput rates.

    The Chicoms demand access to foreign and domestic company computers as a condition of doing business there. How will my cloud intellectual property rights be protected.

    If I like version 5 rather than 6 and I continue to pay maintenance can I still use V5.

    Am I allowed to dictate how new versions are rolled out at my company or will I just show up to a done deal for better or worse.

    Will there be a cloud "beta" version of upcoming rollouts we can use to train staff with before the new version is implemented.

    Will stand alone seats continue to be available with the customary steady improvements we pay for and rightly expect for as long as we wish.

    How much of the yearly maintenance fees we pay for good geometry solving and reliable programs is being diverted to cloud creation which we have no interest in.

    Why are you implementing this cloud when basic geometry creation still has big problems. Are these issues being fixed for the cloud or will we be stuck still with problems and now with the additional layer of problems the cloud will bring to compound our woes.

    If I stop subscribing can I download all my intellectual property for my own purposes.

    If someone hacks the server farm will I A, be notified of this and what was compromised and B, will I be compensated for all resultant damages.

    If I download my files for archival will I be able to use them after creation on the cloud or will I be prohibited the use of my own creations when I stop paying.

    Just how do you intend to charge for this and where is the proof you will save me money here.I will still have to maintain an internal set of workstations and perhaps a network to so where are all the big savings in equipment since I still have to buy what I could independently use away from the cloud to access the cloud.

    How will I be compensated for additional ISP costs since many of them are starting data caps per month based on levels of fees for useage and will increased costs here be offset by decreased cloud costs on a sliding scale since all this cloud stuff is supposed to save me money.

    Who really owns my created data if it resides on your server and I stop paying. Do you hold it hostage for a one time payout to retrieve my data or for the resumtion of monthly payments.

    Since part of what is being touted here is "cad creation anywhere" will I be compensated for increased costs if I begin to use cell phone services and WiFi hotspots to conduct business with.  Please see what are my total costs once again and demonstrate how the cloud will save me money over using my laptop and standalone seat and open new doors for user convenience.

    Intel is coming out with a new series of CPU's with astounding benchmarks priced very cheaply and workstations combined with the new Nvidia cards for example are going to put amazing power on my desktop for under $2,000.00 and this is a one time cost that will usually be good for three or more years. How will you guarantee to me savings here by use of the cloud over what I can do for myself.

    Does the cloud come with field techs to keep this stuff going and implemented correctly or do I still have to retain IT staff. Basically I am asking how will I save money and time with the cloud in this area over what I do now.

    How will you handle a bad version rollout when it creates problems for your customers.

    I live in an area where there will be no great ISP service for some time, Will I be forced to look for other software because you are going to phase out standalone seats and if you are not going to phase out standalone seats will you give me a guarantee of this for a specific time period.

    Will you stop developement of stand alone permanant seat capabilities and direct the developement money to forcing users to migrate to the cloud paradigm.

    Am I going to be able to even create a file if I have no access to the internet for whatever reason and if so to what degree.

    How will you handle data corruption via repeated transmissions over ISP's

    Lets say that one of your translators is failing and I have a critical job to finish. Can I choose which version of the translator I wish to use or is it whatever you care to serve up that day and to bad if it does not work.

    How are you going to convince my customers whom I have confidentiality aggreements signed with that this cloud will be safe to use.

    How will you compensate me for loss of cusomers whose confidentiality aggreements preclude any use of the web or are you just prepared to write me off as not part of the market you are interested in any more.

    How are you going to integrate third party aps into all this.

    Since this cloud is to deliver reliable cost effective services to my company how will you cover me during outages.

    Cyber warfare is now a fact of life. How will you guarantee me service better than an internal implementation that does not go online during cyber attacks or DOS attacks for whatever reason on your cloud implementation and the public infrastructure it has to use to work.

    How will backups of my data be maintained and just who all has access to this proprietary information.

    How will you guarantee that one of your employees will not steal my data and sell it to the highest bidder.

    Will I be able to audit your process to see if it meets my required level of security or do I just have to take your word on this.

    How are you going to make all these companies whose infrastructure you don't own or control give guaranteed minimum levels of service to your customers.

    Many companies wait for a while before implementing new versions and prefer to do this after it has been out for a while and service packs have been issued to correct the problems that are allways there with new versions. How will you allow for this to continue to happen or will you even allow this practice to continue.

  • Deelip,

    From what I understand many cad products today break their files into smaller chunks of data,  something along the lines of body, graphics, history, structure, etc to improve load times. However, as cad vendors move to could db based systems they have significant work to do as the current buckets are far too large for efficiently working solely over the internet to some external cloud. The systems need to store data at a far more granular level such that you could transmit smaller pieces of data more frequently and not perceive any delay as a user (this should also help not losing data) - this seems like a pretty fundamental change for any existing products.

    Dave,

    While some of your questions seem vendor specific, I hope that all the cad vendors take your list to heart – good answers to these questions should be a winning solution. However, the reason these excellent questions have not been answered to date is simply because the vendors don’t know yet themselves. Seems everyone got excited about a technology before figuring out a business plan!

    ps -If you don’t have good bandwidth\connectivity from your isp nothing the cad\could vendors promise is going to change that :)

  • Dave, this article is not about running a CAD application on the Cloud. That is a totally different subject altogether.

  • I think many of dave's questions can  relate to even just storing cad dbs on/in the cloud

  • Many of the questions by Dave Ault are quite legitimate, however, the concept of "private clouds" vs. "public clouds" is something to consider when setting expectations on whether the CAD vendor or the customer is going to be responsible for answering them.

    I am no expert on the "private cloud" concept, but the idea is that a large customer would set up its own "private cloud". A scenario would be Airbus maintaining all data in its private cloud, but if future versions of CAD can be run within a web browser the idea is that all internal users as well as subcontractors would obtain their CAD functionality as well as the CAD data itself, from Airbus. Airbus would have great control over what version(s) of CAD are used, they would have strong  monitoring abilities in terms of product development, etc.

    There are challenges with such a concept, to be sure. But from Airbus perspective, the ability to tightly control the distributed design process would be very appealing. It may not be appealing to the subcontractors, etc.

    The CAD vendors would essentially sell to large companies a "private cloud" implementation. As the large company owns this cloud, many of the questions will need to be addressed by the customer rather than the CAD vendor. Some large companies would be OK with that, since some of them already do this.

    Also, I would concede that much of what I described above can be done now with traditional client / server / network arrangements...I guess the big difference is the part where the CAD tool itself is run within a web browser. HTML5, silverlight and other technologies might allow for this...but I concede that performance, etc. is a big question....but there are very talented developers out there

  • I understand that Dave's questions could be valid for cloud storage as well. I am pretty sure there were people just like Dave who had similar questions when banks started talking about doing banking on the internet. I don't see many people having a problem with that these days. In fact, that's the preferred way of doing banking. Its only a matter of technology and time. And yes, mindset as well.

    To drive my point in, one fine morning your government or a court could freeze your bank accounts. So in fear of that do you stuff all your money in a hole in your back yard? No.

    Dave asks for all kinds of guarantees. What guarantee does Dave have that his government or a court will not freeze his money? None.

    If you want control over everything and not have to be at the mercy of anyone you had better start thinking about living in a cave. ;-)

  • Dassault's endeavor reminds me of MS "WinFS" in Longhorn project.  managing everything in relational db is noble idea, but not sure it can be actually implemented.  Jim Allchin said that improved search in Vista could achieve what WinFS originally envisioned. (windowsteamblog.com/.../working-with-data-in-windows-vista.aspx)

    not sure how CATIA V6 is doing, but i think i will be difficult to be accepted by users whether managed by clouds or not.

  • This is an excellent article.

    One more thing this would do is it would Democratize the entire CAD industry not just manufacturing or architecture. Currently many companies are tied to a particular product or software vendor because all their previous projects are in a particular file format. Moreover, those same software companies are not able to churn out new features because that could involve a "file format change". And if a new version of product involves a file format change everybody who is using it would have to upgrade their software just for reading a file in the new format (eventhough they may not plan to use any of the new features that come with the new release). This is a huge factor (financial) that affects users/organizations when they decide to upgrade their CAD softwares and it has a chain effect.

  • Jay, if CAD data is stored in a database on the cloud, CAD vendors could still find ways to create the version upgrade problem for users. However, if CAD software itself started running on the cloud and made use of the data residing in the cloud then the problem you describe vanishes completely. I mean, there is just one version of Gmail. Everyone uses the same version at any given point in time. Any change in the data format of the emails is taken care of by Google itself. The same thing will happen with CAD software and CAD data as well.

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment