Showing posts with label net. Show all posts
Showing posts with label net. Show all posts

Monday, March 26, 2012

import from excel to SQL Server - can it be more efficient ?

Hello All,

Iam new to the world of ASP .Net. Right now iam building an application that will IMPORT about 5,000 records from an Excel spreadsheet to a table in MS SQL Server. Right now the code works correctly, but i feel it is not efficient and takes a little bit of more time in doing the import. Could you guys throw some light on how i can make the code run more faster ? Someone suggested me that i can use DataAdapter and update the table in the database thru an update method available with it. I dont know how to do it? Could anyone share with me a snippet of code that does this ?

Here is my code:

Private Sub ProcessRecords()
Dim ds2 As New DataSet
' readExcelSheet is a user-defined function that reads a spreadsheet and returns a DataSet object
ds2 = readExcelSheet("C:\Inetpub\wwwroot\Project1\Book2.xls", "SELECT * FROM [Sheet1$]")
Dim myConnection As SqlConnection = Connection() ' user-defined function that returns a SQLConnection object
myConnection.Open()
Dim strSQL As String = "insert_member" ' stored procedure that inserts records
Dim myCommand As New SqlCommand(strSQL, myConnection)
myCommand.CommandType = CommandType.StoredProcedure
myCommand.Parameters.Add("@.salutation", SqlDbType.NVarChar)
myCommand.Parameters.Add("@.firstname", SqlDbType.NVarChar)
myCommand.Parameters.Add("@.lastname", SqlDbType.NVarChar)
myCommand.Parameters.Add("@.company", SqlDbType.NVarChar)

Dim i, j As Integer
Response.Write(Date.Now() & "<br>")
For i = 0 To ds2.Tables("Members").Rows.Count() - 1
myCommand.Parameters("@.salutation").Value = ds2.Tables("Members").Rows(i).Item("sal")
myCommand.Parameters("@.firstname").Value = ds2.Tables("Members").Rows(i).Item("firstname")
myCommand.Parameters("@.lastname").Value = ds2.Tables("Members").Rows(i).Item("lastname")
myCommand.Parameters("@.company").Value = ds2.Tables("Members").Rows(i).Item("company")
j = myCommand.ExecuteNonQuery()
If (j > 0) Then
Response.Write("Record Inserted - " & i + 1 & "<br>")
End If
Next
Response.Write(Date.Now() & "<br>")
myConnection.Close()
End Sub

Please reply soon.
Thank You.can you show us the readExcelObject class?

i got this unspecified exception when trying to connect to an excel file:

System.Data.OleDb.OleDbException: Unspecified error at System.Data.OleDb.OleDbConnectionInternal.InitializeProvider() at System.Data.OleDb.OleDbConnectionInternal..ctor(OleDbConnection conn, OleDbConnectionString constr) at System.Data.OleDb.OleDbConnectionFactory.CreateConnection(DbConnectionOptions options, Object providerInfo, DbConnectionBase owningObject) at System.Data.ProviderBase.DbConnectionFactory.CreateNonPooledConnection(DbConnectionBase owningConnection, DbConnectionPoolGroup poolGroup) at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnectionBase owningConnection) at System.Data.ProviderBase.DbConnectionClosed.Open(DbConnectionBase outerConnection) at System.Data.ProviderBase.DbConnectionBase.Open() at ASP.excel1_aspx.LoadExcelIntoDS() in c:\excel_dev\excel1.aspx:line 25

<%@. Page language="c#" %>
<%@. Import Namespace="System.Runtime"%>
<%@. Import Namespace="System"%>
<%@. Import Namespace="System.Data"%>
<%@. Import Namespace="System.Data.OleDb"%
<script runat=server>
void Page_Load(Object src, EventArgs e)
{
LoadExcelIntoDS();
}
void LoadExcelIntoDS()
{
OleDbConnection MyConnection = new OleDbConnection();
try
{
String conn = "Provider=Microsoft.Jet.OLEDB.4.0;";
conn += @."Data Source=C:\excel_dev\demo_banc.xls;";
conn += "Extended Properties=Excel 8.0;";
MyConnection = new OleDbConnection(conn);
MyConnection.Open();
}
catch(ArgumentException ae)
{
Response.Write(ae.ToString() + "<BR>");
}
catch(OleDbException e)
{
String err_msg = "";
for (int i = 0; i < e.Errors.Count; i++)
err_msg += "------<br>Index #" + i + "<br>Message: " + e.Errors[i].Message + "<br>NativeError: " + e.Errors[i].NativeError + "<br>Source: " + e.Errors[i].Source + "<br>SQLState: " + e.Errors[i].SQLState + "<br>";
Response.Write(err_msg + "<br>");
Response.Write(e.ToString() + "<BR>");
}
catch(Exception e)
{
Response.Write(e.ToString() + "<br>");
}
finally
{
MyConnection.Close();
}
}|||solved.

pls refer to:

http://www.asp.net/Forums/ShowPost.aspx?tabindex=1&PostID=314899

Friday, March 23, 2012

Import Flat file to sql server programmatically

Hi,

i want to import flat file data to sql server. i created a package in vb.net. if the import table column is identity means i got

Failure inserting into the read-only column "ID".
Column metadata validation failed.
"component "OLE DB Destination" (10)" failed validation and returned validation status "VS_ISBROKEN".
One or more component failed validation.
There were errors during task validation. error.

how can i rectify this error? or how can i ignore the identity column in coding.

thanks & regards,

sivani

Don't map a column to the identity column in the table. Leave it out.

For instance, in terms of SQL this won't work:

insert into table (ID, Description) values (1,'Row1');

But this will:

insert into table (Description) values ('Row1');

So in your package, don't map a data flow field to the identity column.|||

Hi,

Thanks for your reply. i want it to do programmatically. Give me some sample code in vb.net to igmore the identity column.

regards,

sivani

|||

Hi,

i have created a package successfully to import flat file. i got some error while i am importing fixedwidth textfiles. The error is,

No column was specified to allow the component to advance through the file.
component "Flat File Source" (1) failed the pre-execute phase and returned error code 0xC02020F0.

and my text file data like this,

00001234561195469 040000 0000 7674491234 47674491234 211252901209030000199804130128431998041301292900003800IXCRD1


00001234568002251234 0400234096321234 0000 8002251234 09782251234 111170401218830000199804130634501998041305345900000400IXCRD1

can anybody give me a solution?

Thanks & regards,

sivani

|||

Did you find an anwer to your question?

Jay

|||

Sivani wrote:

Hi,

Thanks for your reply. i want it to do programmatically. Give me some sample code in vb.net to igmore the identity column.

regards,

sivani

I don't understand. Don't do anything with the identity column. Leave it out/alone.|||

Yes, Finally i found the solution

Jayanthi.

Import Flat file to sql server programmatically

Hi,

i want to import flat file data to sql server. i created a package in vb.net. if the import table column is identity means i got

Failure inserting into the read-only column "ID".
Column metadata validation failed.
"component "OLE DB Destination" (10)" failed validation and returned validation status "VS_ISBROKEN".
One or more component failed validation.
There were errors during task validation. error.

how can i rectify this error? or how can i ignore the identity column in coding.

thanks & regards,

sivani

Don't map a column to the identity column in the table. Leave it out.

For instance, in terms of SQL this won't work:

insert into table (ID, Description) values (1,'Row1');

But this will:

insert into table (Description) values ('Row1');

So in your package, don't map a data flow field to the identity column.|||

Hi,

Thanks for your reply. i want it to do programmatically. Give me some sample code in vb.net to igmore the identity column.

regards,

sivani

|||

Hi,

i have created a package successfully to import flat file. i got some error while i am importing fixedwidth textfiles. The error is,

No column was specified to allow the component to advance through the file.
component "Flat File Source" (1) failed the pre-execute phase and returned error code 0xC02020F0.

and my text file data like this,

00001234561195469 040000 0000 7674491234 47674491234 211252901209030000199804130128431998041301292900003800IXCRD1


00001234568002251234 0400234096321234 0000 8002251234 09782251234 111170401218830000199804130634501998041305345900000400IXCRD1

can anybody give me a solution?

Thanks & regards,

sivani

|||

Did you find an anwer to your question?

Jay

|||

Sivani wrote:

Hi,

Thanks for your reply. i want it to do programmatically. Give me some sample code in vb.net to igmore the identity column.

regards,

sivani

I don't understand. Don't do anything with the identity column. Leave it out/alone.|||

Yes, Finally i found the solution

Jayanthi.

Import Flat file to sql server programmatically

Hi,

i want to import flat file data to sql server. i created a package in vb.net. if the import table column is identity means i got

Failure inserting into the read-only column "ID".
Column metadata validation failed.
"component "OLE DB Destination" (10)" failed validation and returned validation status "VS_ISBROKEN".
One or more component failed validation.
There were errors during task validation. error.

how can i rectify this error? or how can i ignore the identity column in coding.

thanks & regards,

sivani

Don't map a column to the identity column in the table. Leave it out.

For instance, in terms of SQL this won't work:

insert into table (ID, Description) values (1,'Row1');

But this will:

insert into table (Description) values ('Row1');

So in your package, don't map a data flow field to the identity column.|||

Hi,

Thanks for your reply. i want it to do programmatically. Give me some sample code in vb.net to igmore the identity column.

regards,

sivani

|||

Hi,

i have created a package successfully to import flat file. i got some error while i am importing fixedwidth textfiles. The error is,

No column was specified to allow the component to advance through the file.
component "Flat File Source" (1) failed the pre-execute phase and returned error code 0xC02020F0.

and my text file data like this,

00001234561195469 040000 0000 7674491234 47674491234 211252901209030000199804130128431998041301292900003800IXCRD1


00001234568002251234 0400234096321234 0000 8002251234 09782251234 111170401218830000199804130634501998041305345900000400IXCRD1

can anybody give me a solution?

Thanks & regards,

sivani

|||

Did you find an anwer to your question?

Jay

|||

Sivani wrote:

Hi,

Thanks for your reply. i want it to do programmatically. Give me some sample code in vb.net to igmore the identity column.

regards,

sivani

I don't understand. Don't do anything with the identity column. Leave it out/alone.|||

Yes, Finally i found the solution

Jayanthi.

Monday, March 19, 2012

Import database from access

Hi
I am develop a project with .net in windows application.my source of the
database set in access I want to working with WMSDE and I need to import the
database under the access to database in WMSDE first Is it passible?
If yes, how can I do this.
Thanks for who refer to my question
doesn't access have an upsizing wizard in it ?
"amos hchmon" <amoshchmon@.discussions.microsoft.com> wrote in message
news:91853F12-5194-4D31-891F-0C5EAFC16E94@.microsoft.com...
> Hi
> I am develop a project with .net in windows application.my source of the
> database set in access I want to working with WMSDE and I need to import
> the
> database under the access to database in WMSDE first Is it passible?
> If yes, how can I do this.
> Thanks for who refer to my question

Import database from access

Hi
I am develop a project with .net in windows application.my source of the
database set in access I want to working with WMSDE and I need to import the
database under the access to database in WMSDE first Is it passible?
If yes, how can I do this.
Thanks for who refer to my questiondoesn't access have an upsizing wizard in it ?
"amos hchmon" <amoshchmon@.discussions.microsoft.com> wrote in message
news:91853F12-5194-4D31-891F-0C5EAFC16E94@.microsoft.com...
> Hi
> I am develop a project with .net in windows application.my source of the
> database set in access I want to working with WMSDE and I need to import
> the
> database under the access to database in WMSDE first Is it passible?
> If yes, how can I do this.
> Thanks for who refer to my question|||you can import the access via SQL Servers import function in the EMC.

Import database from access

Hi
I am develop a project with .net in windows application.my source of the
database set in access I want to working with WMSDE and I need to import th
e
database under the access to database in WMSDE first Is it passible?
If yes, how can I do this.
Thanks for who refer to my questiondoesn't access have an upsizing wizard in it ?
"amos hchmon" <amoshchmon@.discussions.microsoft.com> wrote in message
news:91853F12-5194-4D31-891F-0C5EAFC16E94@.microsoft.com...
> Hi
> I am develop a project with .net in windows application.my source of the
> database set in access I want to working with WMSDE and I need to import
> the
> database under the access to database in WMSDE first Is it passible?
> If yes, how can I do this.
> Thanks for who refer to my question

Monday, March 12, 2012

import data from Text Files into SQL Server...?

Hi,
i wanna develop an web-database application with ASP.NET,C#, SQL server 2000.
i already have some data whichs been in text format(text file) and now, i want to import the same into my database.
the problem is, the text file has got many line breaks and also its not well formated to import it using DTS.
Can any one help me out in importing the same.
thanks in advance

Hello Kiran,
There is no way to transform the text file if it is not formatted and inconsistent.
Only thing you can do is to make it consistent at application end and then passes it to DTS package.
|||Try this link see if you can use BCP (bulk copy) with DTS. Hope this helps.
http://www.sqldts.com/default.aspx?237|||Hi,
Thanks for the response.
Even i think i need to format the text in the front end, before actually using the DTS.And thus formatted one can be easily imported.Well i gonna try that, but the probs is , even if i want to format it in the front end, the same cant be generalized to all, since each of them gonna have there own format.
any more ideas are always welcome

Import Data from SPSS (Statistical Package for the Social Sciences

hi to everyone. I'm developing a web application using C#.NET and MS SQL
Server 2000 database backend. But the user wants to transfer a huge data
from his SPSS application because it will take large amount of time to input
each of the item. My question is: Is there a way to import data from SPSS
application to the database using MS SQL Server 2000.
Specifications:
Import Data From a data file (SPSS Standard Version 11.0.0)
Hope you'll reply as soon as possible. You idea will greatly help me.
Thank you and God Bless.Hi
If this information is in a reasonably formatted file you can use BULK
INSERT command, DTS or the BCP utility to import the information. This can
be scheduled as a job and you can archive the file once loaded.
There is plenty of information on BULK INSERT, DTS and BCP in Books Online,
also check out http://www.sqldts.com/ for more DTS articles such as
http://www.sqldts.com/default.aspx?231
and http://www.sqldts.com/default.aspx?246
You will need some method of loading the data file onto your server or
somewhere accessable from the server. It may be necessary to hold the data
in a staging table(s) if it requires additional work before loading into
your destination table(s).
John
"rolly-hubport" <rollyhubport@.discussions.microsoft.com> wrote in message
news:231A4F51-B185-44D8-AE52-730275EC3C21@.microsoft.com...
> hi to everyone. I'm developing a web application using C#.NET and MS SQL
> Server 2000 database backend. But the user wants to transfer a huge data
> from his SPSS application because it will take large amount of time to
> input
> each of the item. My question is: Is there a way to import data from SPSS
> application to the database using MS SQL Server 2000.
> Specifications:
> Import Data From a data file (SPSS Standard Version 11.0.0)
>
> Hope you'll reply as soon as possible. You idea will greatly help me.
> Thank you and God Bless.
>
>|||Hi,
If u will use BCP or Bulk insert u have to manually define the table
name.
use dts package tranfer data from one server to another and u have
option to select the tables.
hope this helps u
from
doller

import data from ODBC to SQL2005

Hi, there;
I want to importing data from ODBC, I created DataReader Source which use a .NET Provide \Odbc Data Provider and connected successfully. My destination is a OLE DB Destination that points to SQL2005. I set the SQL command as "SELECT * from ....".
I also have problem to create new table in SQL2005 using SSIS Import and Export Wizard, it doesn't know the source table table schema (two date type column). So I create the new table manully and run the package, I got error:

SSIS package "Package1.dtsx" starting.
Information: 0x4004300A at Data Flow Task, DTS.Pipeline: Validation phase is beginning.
Information: 0x40043006 at Data Flow Task, DTS.Pipeline: Prepare for Execute phase is beginning.
Information: 0x40043007 at Data Flow Task, DTS.Pipeline: Pre-Execute phase is beginning.
Information: 0x4004300C at Data Flow Task, DTS.Pipeline: Execute phase is beginning.
Error: 0xC02090F5 at Data Flow Task, Source - Query [1]: The component "Source - Query" (1) was unable to process the data.
Error: 0xC0047038 at Data Flow Task, DTS.Pipeline: The PrimeOutput method on component "Source - Query" (1) returned error code 0xC02090F5. The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing.
Error: 0xC0047021 at Data Flow Task, DTS.Pipeline: Thread "SourceThread0" has exited with error code 0xC0047038.
Error: 0xC0047039 at Data Flow Task, DTS.Pipeline: Thread "WorkThread0" received a shutdown signal and is terminating. The user requested a shutdown, or an error in another thread is causing the pipeline to shutdown.
Error: 0xC0047021 at Data Flow Task, DTS.Pipeline: Thread "WorkThread0" has exited with error code 0xC0047039.
Information: 0x40043008 at Data Flow Task, DTS.Pipeline: Post Execute phase is beginning.
Information: 0x402090DF at Data Flow Task, Destination - PITest CMF [169]: The final commit for the data insertion has started.
Information: 0x402090E0 at Data Flow Task, Destination - PITest CMF [169]: The final commit for the data insertion has ended.
Information: 0x40043009 at Data Flow Task, DTS.Pipeline: Cleanup phase is beginning.
Information: 0x4004300B at Data Flow Task, DTS.Pipeline: "component "Destination - PITest CMF" (169)" wrote 0 rows.
Task failed: Data Flow Task
SSIS package "Package1.dtsx" finished: Success.

Can any one know what's wrong here?

And it is quite pain that you have to specify the table name every time you want to create a new table in SQL2005. It is so easy in SQL2000!!!

Thanks.Did you verify that your source query worked and the column mappings metadata is correct? That's where I would start looking.

Friday, March 9, 2012

Import csv files to Sql Server problem

Hi,

I try to import csv files to Sql Server using .net. The code is as following:

string strCsvConn =@."Provider=Microsoft.Jet.OLEDB.4.0;Data Source=\;Extended Properties='text;HDR=Yes;FMT=Delimited(,)';";

using (OleDbConnection cn =newOleDbConnection(strCsvConn))

{

string strSQL ="SELECT * FROM " + strFileName;

OleDbCommand cmd =newOleDbCommand(strSQL, cn);

cn.Open();

using (OleDbDataReader dr = cmd.ExecuteReader())

{

// Bulk Copy to SQL Server

using (SqlBulkCopy bulkCopy =newSqlBulkCopy(strSqlConn))

{

bulkCopy.DestinationTableName = strSqlTable;

bulkCopy.WriteToServer(dr);

}

}

}

And the data is as following (simplified):

Model,Serial

AFICIO 3045,K9464900965

AFICIO 3045,K9464900932

Fax 5510L,A3761290041

Fax 2210L,A4978800008

AFICIO 3025,K8565201014

AFICIO 3025,K8565102398

The result of the 2nd column is: 9464900965, 9464900932, null, null, 8565201014, 8565102398 - either the first character is missing or the whole entry is missing.

One more weird thing is that some other files work fine, though I am not able to tell any difference between them.

Any idea is hoghly appreciated.

shz

Hmmm... what's the datatype / size of the second column in your destination database?

In your example, the two rows that get null in second column contains a space in the first column. Is it maybe because space is used as a separator too in some way?|||

Thanks johram,

The datatype is varchar(50). However, I am afraid it has nothing to with the database, because it is the DataReader that retrieves wrong data. I test the DataReader with the following code:

while (dr.Read())

{

string str =Convert.ToString(dr[1]);

}

And all entries in the 1st column contain a space. The file contains more than 10 columns actually, all other columns are good.

Some more findings:

I have some "good" files that work fine and some "bad" files that have this problem - I cannot tell any difference between them in terms of data format. If I copy some records from a "good" file to a "bad" file, those records become bad. If I copy some records from a "bad" file to a "good" file, those records become good.

If I use TDS to import the files to SQL Server, it works fine. Excel can open the files properly too.

Thanks,

shz

|||

It seems to be the problem in the header (columns) of the cvs file.

Please make sure thay are okay.

Good luck.

|||

Fixed - need a Schema.ini file to define the Extended Properties of the driver.

Thanks to everyone.

Wednesday, March 7, 2012

Import csv file to MS SQL 2005

Hi Guys,

I have been trying to search for a free asp or asp.net script that will allow me to upload a .csv file and import it into an MS SQL Database. As its going to be a ProductCatalog and pricing changes nearly 2nd day. And wanting to an a script that I can put in my admin panel on my site to upload a .csv file and import it to a MS SQL Database.

I will be updating fields as well as adding new products. So the upload script would need to be able to handle those two things.

Is their any good free scripts around that people can recommend.

Thanks

Matthew

Which version of SQL server are you using?

SQL Server Integration Services will do this nicely...

|||

Using SQL 2005 Standard Edition, as the database will be used on a Website, don't want to have to keep logging into the control panel and then going and using the Web-Based SQL Management Tools.

Matthew

|||

Hello,

http://www.nigelrivett.net/ImportTextFiles.html is a script to import text files that arrive in a directory into a table.

It will process every file in the directory with the correct filemask and move the file to an archive directory on completion.
It can be used in conjunction with an ftpget SP to import files from a ftp server .

You could add some minor change to import csv file.

Friday, February 24, 2012

Import Access Report

I need to import Access Reports to Reporting Service on fly using VB.NET code. How can I do that? I know how to do that using Report designer but it is not going to work because we have to convert 500 Crying [:'(]reports. Any help will be highly appreciated.

I'm in the same boat. I have tried various report-to-xml export options but I get nothing but errors trying to get a functioning xml report working in Visual web developer. Sure hope someone can chime in on this subject. Zero help from Microsoft.

Import a database with .net framework data provider for ODBC

mHello all,
i have to import a database (micro focus files with idx-files) with an obdc
driver. When I use the assistant, I choose ".net framework data provider for
ODBC". In the next window I am asked for the connection string an driver. How
can I specify the driver to be used and the location of the original
databaseIn SQL 2000 it was quite easy using the ODBC Connection manager, but
in SQL 2005 I cant find the drivers. Any help or link to a documentation
appreciated. Thank you.
I have the same problem, did you ever find the fix for this?
i have to import a database (micro focus files with idx-files) with an obdc
driver. When I use the assistant, I choose ".net framework data provider for
ODBC". In the next window I am asked for the connection string an driver. How
can I specify the driver to be used and the location of the original
databaseIn SQL 2000 it was quite easy using the ODBC Connection manager, but
in SQL 2005 I cant find the drivers. Any help or link to a documentation
appreciated. Thank you.[/quote]

Import a database with .net framework data provider for ODBC

mHello all,
i have to import a database (micro focus files with idx-files) with an obdc
driver. When I use the assistant, I choose ".net framework data provider for
ODBC". In the next window I am asked for the connection string an driver. Ho
w
can I specify the driver to be used and the location of the original
databaseIn SQL 2000 it was quite easy using the ODBC Connection manager, but
in SQL 2005 I cant find the drivers. Any help or link to a documentation
appreciated. Thank you.I have the same problem, did you ever find the fix for this'
i have to import a database (micro focus files with idx-files) with an
obdc
driver. When I use the assistant, I choose ".net framework data
provider for
ODBC". In the next window I am asked for the connection string an
driver. How
can I specify the driver to be used and the location of the original
databaseIn SQL 2000 it was quite easy using the ODBC Connection
manager, but
in SQL 2005 I cant find the drivers. Any help or link to a
documentation
appreciated. Thank you.
RCranston