What Is The Best Practice To Handle Cobol Redeines In Flat File Data.

Mar 1, 2007

Here is my idea but I am looking for the best practice.

Each record can have 3 possibilites.

I would read and write the data 3 times to different tables and add an identity key on all 3 files then I would reassembe the data back together on the identity key and map the data to a fourth and final table.

I am a novice in data base designing. I have MS certification in both design, implementation and administration. Can anyone help me provide information regarding converting from Cobol flat files to sql server tables ?I will be very grateful.. This is my first assignment.. Got to prove myself... help....

Hi All,I need to send out email when error occurs in the package. Is it a good practice to put the send email task in the event handler? Then MaximumErrorCount is set to 1. But for some reason, some time I saw more than one email are sent out. Please advise. Thanks

I am testing SSIS and have created a Flat File Destination. I defined the Flat File Connection as New for the first time and it worked fine. Now, I would like to go back and modify the Flat File Connection in the Flat File Destination Editor, but it allows only to create a New connection rather allowing me to edit the existing one. For testing, I can go back and create a new connection, but if my connection had 50-100 columns then it would be an issue to re-create it from scratch.

In a foreachloop, I am inserting records into a flat file which is working fine. But the thing is that as the file grows, it takes longer for it to locate the EOF(End of File) of the flat file so as to insert the records.

I have around 70-100 lines written to the file at each loop and there are more than 20k records to be looped. wihich means that at the end I should be having 1400k - 20000k line in the text file.

One solution would be to insert the records at the start of the file itself so that it does not has to lookup the EOF each time before writting.

I have a situation where a tab limited text file is used to populate a sql server table.

The tab limited text file comes from a third party vendor. There are fixed number of columns we need to export to the sql server table. However the third party may add colums in the text file. Whenenver the text file has an added column (which we dont need to import) the build fails since the flat file connection manager does not create the metadata for it again. The problem goes away where I press the button "Reset Columns" since it builds the metadata then. Since we need to build the tables everyday we cannot automate it using SSIS because the metadata does not change automatically. Is there a way out in SSIS?

I am transferring data from an OLEDB source to a Flat File Destination and I want the column width for all of the output columns to 30 (max width amongst the columns selected), but that is not refected in the Fixed Width Flat File that got created. The outputcolumnwidth seems to be the same as the inputcolumnwidth. Is there any other setting that I am possibly missing or is this a possible defect?

I am running my package in sql server 2012, in which i am giving network path for flat file destination. And its working fine. But if i give m local path, its giving me error " cannot open data file" ...

How do I insert data from a flat file or .csv file into an existing SQL database???

Here what I've come up with thus far and I but it doesn't work. Can someone please help? Let me know if there is a better way to do this... Idealy I'd like to write straight to the sql database and skip the datset all together...

How do I insert data from a flat file or .csv file into an existing SQL database???

Here what I've come up with thus far and I but it doesn't work. Can someone please help? Let me know if there is a better wway to do this... Idealy I'd like to write straight to the sql database and skip the datset all together...

Is there a better way to handle fixed width flat files than the built-in SSIS capability? I have a fixed width file with over 400 columns and it looks like I need to manually click lines where each column starts/ends (quite tedious and prone to error). I have an excel version of the spec with start position, length, and data type for each column. So far it looks like the only way to automate this task is to somehow automatically generate the package XML from the spec and paste it into the dtsx file. Anyone know of a better way?

I m using SSIS and i am transfering the data from Flat File Source to the OLE DB destination File. The source file contain some corrupt data which i am transfering to the other Flat file destination file.

Debugging is succesful but i am not getting any error output in the Flat file destination file.

i had done exactly which is written in the msdn tutorial of SSIS.

Plz tell me why i am not getting the error output in the destination flat file?

I have to value [CreateDate] in the data pump of my Flat File Source into my OLE DB Destination SQL Server Table. With a Variable within the SSIS Package or with a Derived Column task within the Data Flow between the Flat File Source and OLE DB Destination?

I'm moving data between identical tables and have to use a flat file as an intermediary. I thought: "No problem, SSIS can do a quick export to a file, then move the file to another server, then use SSIS to import the data to the new server."

Seems simple, right?

I'm hitting all sorts of surprising data conversion errors. I used the export wizard to create the export package. This works fine. However using the same flat file definition, the import package fails -- even when I have no destination. That is I have just one data flow task that contains only one control: the Flat File source. When I run the package the flat file definition fails with data type conversion and truncation errors. One of the obvious errors is for boolean types. The SQL field is a bit, SSIS defined the column as DT_BOOL, the output of the data are literal text values "TRUE" and "FALSE". So SSIS converts a sql datatype of bit to "TRUE" and "FALSE" on export, but can't make the reverse conversion on import?

Does anyone else find this surprising? I would expect that what SSIS exports, it can import given all the same table and flat file definitions. Is SSIS the wrong tool to do such simple bulk copies? I'd like to avoid using BCP because this process will need to run automatically within SQL Agent so we can leverage all the error tracking and system monitoring.

First, a couple of important bits of information. Until last week, I had never touched SISS, and therefore, I know very little about it. I just never had the need to use it...until now. I was able to convert my first 3 flat files to SQL2005 tables by right clicking on "SISS Package" and choosing "SISS Import and Export Wizard". That is the extent of my knowledge! So please, please, please be patient with me and be as descriptive as possible.

I thought I could attach some sample files to this post, but it doesn't look like I can. I'll just paste the information below in two separate code boxes. The first code box is the flat file specifications and the second one is a sample single line flat file similar to what I'm dealing with (the real flat file is over 2 gigs).

As you can see, each entry could have any number of records and multiples of some of the record types, with one exception, every entry must have a "01" record and can only have one "01" record. Oh, and each record has a length of 400.

I need to get this information into a SQL 2005 database so I can create a front end for accessing the data. Originally, I wanted one line for each account and have null values listed for entries that don't have a specific record. Now that I've looked at the data again, that doesn't look like a good idea. I think a better way to do it would be to create 5 different tables, one for each record type. However, records 2 through 5 don't have anything I can make a primary key. So here are my questions...

Is it possible to make 5 tables from this one file, one table for each of the record types?

If so, can I copy the Account number in record 01, position 13-19 in each of the subsequent record types (that way I could link the tables as needed)?

Can this be done using the SISS Import and Export Wizard to create the package? If not, I'm going to need some very basic step by step instructions on how to create the package.

Is SISS the best way to do this conversion or is there another program that would be better to use?I know this is a huge question and I appreciate the help of anyone who boldly decides to help me! Thank you in advance and I welcome anyone's suggestions!

1. I am attempting to export data from a SQL DB (single table using a query) to a "flat file". 2. I would then like to take this "flat file" and import the data into a different SQL DB (same schema structure as first DB).

I was wondering if there was a way for me to append data to a flat file. The reason why i ask this is because i need to create a header for the report that i am exporting.

The way i imagined this working would to be create a dts that would export the header information to a flat file and the create another dts to export the report data and appensd it to the same file that the header dts created. This might not be the correct approach so i was hoping i could get some guidance of how i can accomplish this.

i am currently creating a package which involves getting data from CSV files. i can successfully get the data from the files, my problem is, i need to get data from the header of the CSV files. i am currently skipping the header rows. the format of the CSV files is as follows:

technically, i also need the date from the header row, but since it is also indicated in the data rows, i have no problem with that. what i need is the Store Code, which is not indicated on the data rows. i need to store the data in a database in the following format:

I am looking to import data into SQL Server 2005 using SSIS. I want to take data that is contained in a flat file and place it into the various appropriate tables in my system. The flat file contains nested data. For example...

Order Table: OrderID, Amount (ID 123 @ $3.33 and another row ID 245 @ $1.99). If possible I would also like to tie the orders to the client record that was inserted.

My first question is if SSIS supports nested fields as in my example. Can it break a file by commas, then within a field by other delimiters? If so how do I do this, and if not what is the recommend way to accomplish this sort of task.

My second quesiton is if it can do that, can it tie the Client and Order data on the fly?

any suggestions on dealing with a flat file in the format below. I only want to process the data columns in the middle of the file and want to ignore all other rows. This was a very simple task in DTS with a small amount of VBScript in the transformation but it doesn't seem as straightforward in SSIS. thanks

I wanted to know if there was a way to import data from a flat file without specifiying the delimiters. I want to import each line in one row so that i can use the substring function to break of the data as an when i want and not as per the delimited format file or the wizard.

i.e if row one had "abc"|"1453"|"Jack"|"Smith"| etc.... rather than importing these as different columns and rows. I want this all in one row, one column.

Hi-I have a sql 2005 server (New to me) that I need to get a report out of of type .inputIt needs to be in the format: HeaderRowreportTitile + rowcount()+ DateFixedWidthcol1 (spaces pad this field to 25) FixedWidthCol2FixedWidthcol3..... FixedWidthcol1 (spaces pad this field to 25) FixedWidthCol2FixedWidthcol3..... The columns in the flatfile HAVETO run into eachother, with NO delimiters.If i try and export it as a fixedWidth file, the columns DON'T line up. If I try as a delimited file, I can't do it w/o having a delimiter character in between.I'm creating an SSIS file (New to me) to do this, is this my best shot of getting a flat text file that I can then ftp to another server? thanksDan